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ABSTRACT 


Automated  testing  is  an  engineering  tool  used  to  test  designs  by  exposing  them 
to  a  variety  of  conditions  and  combination  of  conditions  in  an  efficient  manner.  While 
guidelines  exist  for  designing  human  computer  interfaces  (HCIs)  or  in  the  special 
case  of  aviation,  the  pilot  vehicle  interface  (PVl),  testing  these  interfaces  usually  re¬ 
quires  extensive  human-in-the-loop  (HITL)  testing.  An  alternative  to  HITL  testing  is  to 
model  human  computer  interaction  in  software  and  employ  the  model  as  an  automated 
cognitive  engineering  tool  for  testing  HCI/PVI  design  for  the  purpose  of  understanding 
the  impacts  on  human  performance  and  mission  effectiveness. 

The  present  study  used  computer  simulation  to  compare  the  effectiveness  of  an 
existing  and  proposed  PVl  for  air-to-ground  strike  missions  employing  the  Maverick 
missile.  The  baseline  Interface  involved  using  Forward-Looking  Infrared  Radar 
(FUR)  to  detect  and  designate  targets.  The  improved  PVl  enhanced  FUR  contrast  and 
added  Real-Time  information  in  the  Cockpit  (RTIC)  overhead  imagery  of  the  target 
area  with  targets  marked.  These  PVIs  had  been  previously  tested  by  HITL  simulations 
and  several  software  models.  Keeping  this  effort  blind  to  the  results  of  the  previous  ef¬ 
forts,  the  Air  Force  only  provided  task  analyses  and  measures  used  in  the  earlier  efforts 
so  comparisons  could  be  made. 

The  Soar  software  architecture  was  chosen  to  model  pilot  cognition  because  of 
its  human-like  cognitive  capabilities.  As  it  turned  out,  target  acquisition  for  this  particu¬ 
lar  mission  scenario  was  more  dependent  on  the  parameters  chosen  for  the  pilot’s  vis¬ 
ual  and  motor  functions  than  cognitive  processing.  Hovyever,  the  Soar  results  pre¬ 
sented  here  were  comparable  to  HITL  data  and  the  other  cognitive  modeling  simula¬ 
tions.  Even  though  the  visual  and  motor  parameters  chosen  for  the  Soar  model  had 
limited  correlation  to  the  HITL  data,  the  Soar  model  did  accurately  predict  faster  target 
acquisition  for  the  RTIC  PVl  and  faster  target  acquisition  for  sparser  scene  density. 
Although  not  statistically  significant,  the  Soar  model  correctly  indicated  that  increased 
scene  complexity  would  have  a  greater  Increase  on  target  acquisition  for  the  RTIC  PVl 
condition  as  compared  to  the  baseline  condition  (HITL  179%  increase.  Soar  47%  in¬ 
crease).  However,  the  previous  models  failed  to  indicate  the  increased  target  acquisi- 
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tion  times  for  increased  scene  complexity  for  the  RTIC  condition  when  compared  to  the 
baseline  condition  (Cognitive  Task  Anaiysis  (CTA)  105%  decrease,  Traditionai  Task 
Anaiysis  (TTA)  124%  decrease). 
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INTRODUCTION 


A  growing  number  of  accidents  and  incidents  can  be  directly  or  indirectly  attrib¬ 
uted  to  mental  errors  made  by  human  system  operators  (O’Hare,  Wiggins,  Batt,  and 
Morrison,  1994).  These  errors  are  particularly  disturbing  to  the  engineers  who  build 
these  systems  because  the  hardware  and  software  all  worked  as  intended,  but  the  re¬ 
sult  was  still  tragic  failure.  Clearly,  there  is  a  pressing  need  to  be  able  to  predict  the  cir¬ 
cumstances  leading  to  and  ultimately  diminishing  the  possibility  of  such  mental  errors. 
One  approach  to  predicting  such  errors  is  to  perform  high-fidelity,  HITL  simulations  and 
observe  where  cognitive  breakdowns  appear.  Unfortunately,  there  are  drawbacks  to 
this  approach  which  make  thorough  testing  impractical.  The  combination  of  high-fidelity 
simulation  costs,  limits  in  skilled  operator  availability,  behavior  differences  between 
simulation  and  real  flying,  and  the  limitless  combinations  of  events  capable  of  triggering 
cataclysmic  accidents  make  thorough  HITL  testing  impractical. 

An  alternative  approach  to  HITL  testing  is  to  model  the  operator’s  cognitive  per¬ 
formance,  subject  the  model  to  the  broadest  range  of  possible  circumstances,  and  rec¬ 
ord  where  the  cognitive  failures  occur.  This  is  analogous  to  automated  testing  which  is 
already  extensively  used  by  engineers  to  increase  product  reliability.  Although  model¬ 
ing  human  cognition  is  a  fledgling  science,  modeling  methods  do  exist,  such  as  ACT* 
and  Soar,  which  can  serve  as  the  basis  for  automated  testing.  Cognitive  automated 
testing  of  user  interfaces  has  both  advantages  and  disadvantages.  On  the  down  side, 
automated  cognitive  testing  lacks  the  face  validity  of  HITL  and  the  quality  of  the  out¬ 
come  is  dependent  on  the  quality  and  scope  of  the  model.  However,  such  testing  can 
investigate  large  numbers  of  circumstances,  especially  unusual  circumstances,  at  a 
lower  cost  and  in  less  time.  Furthermore,  models  can  be  readily  changed  and  adapted 
as  additional  information  and  test  runs  become  available.  This  kind  of  modeling  could 
be  used  to  predict  which  unusual  circumstances  cause  these  low  frequency  mental  er¬ 
rors.  Cognitive  modeling  could  become  a  primary  tool  for  designing  and  evaluating 
user  interfaces,  and  potentially  decrease  the  occurrence  of  metal  errors. 
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Architecture  Selection 


Marshak,  Pohlenz  (now  Wesler),  and  Smith  (1996)  considered  how  cognitive 
modeling  could  be  employed  in  the  crewstation  design  process  and  speculated  on  how 
best  to  implement  such  a  model  for  Armstrong  Laboratory’s  CCDT  Program.  A  survey 
covered  the  existing  cognitive  modeling  literature  and  identified  the  bases  for  a  predic¬ 
tive  aircrew  cognitive  model.  The  literature  revealed  two  leading  candidates  in  which  to 
implement  the  cognitive  model.  The  first  is  Anderson’s  ACT*  (1983)  architecture 
(current  version  ACT-R)  which  grew  out  of  cognitive  psychological  theory;  and  Laird, 
Rosenblum,  and  Newell’s  (1985)  State,  Operator,  and  Result  (Soar)  architecture  based 
on  an  artificial  intelligence  architecture  which  mimics  human  performance.  The  follow¬ 
ing  explanation  borrows  heavily  from  Marshak,  et  al.  (1996). 

Anderson’s  (1983)  ACT*  mode!  for  cognition  tries  to  provide  a  comprehensive 
explanation  for  high-level  cognitive  control  behavior.  There  are  three  distinct  memory 
structures  in  Anderson's  model:  working,  declarative,  and  production.  Working  mem¬ 
ory  (WM)  holds  relevant  information  from  declarative  memory,  sensory  inputs,  and  the 
actions  of  productions  (explained  in  the  following).  Declarative  memory  (DM)  contains 
a  tangled  hierarchy  of  information  clusters,  each  with  no  more  than  five  elements.  Pro¬ 
duction  memory  (PM)  contains  IF-THEN  condition-action  pairs  with  conditions  drawn 
from  DM  and  actions  inserted  in  WM. 

Anderson’s  theory  is  based  on  empirical  findings  of  human  learning  and  memory 
studies  and  as  such,  has  been  elaborated  to  match  the  theoretic  thinking  in  that  do¬ 
main.  This  introduces  elaborations  as  separating  memories  into  temporal,  spatial,  and 
abstract  classes.  ACT*  offers  six  kinds  of  interactions  in  WM  and  five  different  mecha¬ 
nisms  for  conflict  resolution.  Although  these  theoretic  elaborations  match  the  complex¬ 
ity  of  human  cognitive  processing  theory,  they  also  increase  parameterization  and  allow 
multiple  predictions  based  on  which  combinations  of  processes  are  operating. 

A  somewhat  simpler  human  cognitive  modeling  scheme  was  proposed  by  Card, 
Morgan,  and  Newell  (1983).  The  Model  Human  Processor  (MHP)  was  a  general 
model  for  human  problem-solving.  MHP  postulated  three  interacting  subsystems 
(perceptual,  motor,  and  cognitive)  each  with  their  own  processor  and  memories.  Per- 
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ceptual  stores  include  visual  arid  auditory  memories.  Motor  stores  have  the  ability  to 
store  sequences  of  motor  behaviors.  Cognitive  stores  include  working  and  long-term 
memory.  Operation  of  the  MHP  model  is  based  on  the  Recognize-Act  cycle,  where 
productions  are  drawn  from  long-term  memory  into  WM  and  the  results  of  their  firing 
modifying  WM.  Operations  are  governed  by  the  principles  of  discrimination  (retrieval 
based  on  uniqueness),  uncertainty  (decision  processes  slow  with  uncertainty),  and  ra¬ 
tionality  (goal-orientation). 

The  MHP  model  has  evolved  into  what  is  now  called  Soar.  Soar  is  an  advanced 
expert  system  software  architecture  which  employs  a  decision  engine  modeled  after 
human  problem-solving.  However,  Newell  (1990,1992)  asserts  that  Soar  is  a  candidate 
for  a  unified  theory  of  human  cognition  instead  of  being  just  another  computational  ex¬ 
pert  system.  Newell  supports  his  claim  based  on  a  common  mental  chronometery 
(similarities  in  timing),  a  common  architecture  with  perception  and  motor  function,  and  a 
convergence  of  Soar's  behavior  with  data  on  human  operations  and  learning.  Newell 
eloquently  supports  the  relationship  between  Soar  and  human  cognition  by  pointing  out 
Soar  can  predict  or  explain  many  human  learning  and  memory  phenomenon. 

Under  Newell’s  urging.  Soar  incorporated  human  intelligence  strategies  as  part 
of  its  software  architecture.  The  first  feature  added  was  universal  sub-goaling  as  a 
means  of  overcoming  impasses.  If  Soar  cannot  initially  achieve  a  solution  to  a  problem, 
it  defines  more  attainable  sub-goals  which  are  attainable.  Once  these  sub-goals  are 
attained,  the  final  goal  is  tested  again  and  if  not  yet  achieved,  new  sub-goals  are  de¬ 
fined.  This  process  continues  until  the  ultimate  goal  can  be  attained  from  the  last 
achieved  sub-goal.  Universal  sub-goaling  circumnavigates  impasses  without  arbitrary 
assumptions  or  ineffective  trial-and-error  processes. 

Another  important  feature  of  Soar  is  the  use  of  chunking  as  a  learning  process 
during  problem-solving.  Chunking,  the  phenomenon  reported  by  Miller  (1959)  that 
overcomes  the  limitations  of  short-term  or  WM,  allows  combination  of  IF-THEN  produc¬ 
tion  rules  into  related  groups.  The  chunking  hypothesis  is  that  "A  human  acquires  and 
organizes  knowledge  of  the  environment  by  forming  and  storing  expressions,  called 
chunks,  which  are  structured  collections  of  the  chunks  existing  at  the  time  of  learning” 
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(pg.  142;  Rosenbloom  in  Laird,  Rosenbloom,  and  Newell,  1986).  It  uses  a  stimulus- 
response  relationship  between  contents  of  chunks  and  follows  from  human  learning 
theory.  Chunking  enables  Soar  to  manage  the  complexity  of  large  production  rule 
bases  by  organizing  them  into  chunks  of  increasing  complexity.  Rosenbloom  goes  on 
to  show  how  Soar  behavior  seems  to  mimic  human  performance  on  memory  and  prob¬ 
lem-solving  tasks. 

The  Soar  approach  has  certain  advantages  that  favored  its  application  in  simula¬ 
tions  over  ACT-R.  First,  Soar  is  an  extremely  parsimonious  approach  to  cognitive 
modeling;  lower  model  complexity  means  a  more  deterministic  approach.  Perhaps 
even  more  important  is  that  Soar  is  already  being  employed  in  modeling  aircrew  behav¬ 
iors.  The  Defense  Advanced  Research  Project  Agency  (DARPA)  funded  both  Drs. 
Laird  and  Rosenbloom  to  develop  intelligent  forces  (IFORs)  for  the  Synthetic  Thea¬ 
ter  of  War  (STOW)  combat  simulation.  The  Soar  architecture  had  already  been  ported 
into  the  Modular  Semi-Automated  Forces  (ModSAF)  software  system.  ModSAF  pro¬ 
vides  aircraft  flight  dynamics  and  weapons  models  while  Soar  provides  the  intelligence 
to  “fly"  IFOR  aircraft  autonomously.  By  using  Soar,  this  substantial  existing  DARPA  ef¬ 
fort  could  be  leveraged  to  create  a  working  model  with  a  minimum  of  time  and  effort. 
Another  advantage  to  the  Soar-ModSAF  approach  is  that  it  is  distributed  interactive 
simulation  (DIS)  compatible.  Models  developed  using  Soar-ModSAF  could  be  tested 
in  complex  DIS  scenarios  and  improvements  made  by  cognitive  modeling  could  en¬ 
hance  Soar  performance  in  STOW. 

Background 

The  CCDT  Program  had  already  conducted  a  HITL  simulation  and  created  two 
different  models  to  predict  the  effectiveness  of  cockpit  display  and  control  changes  In 
the  F-15  based  fighter  and  its  ability  to  deliver  multiple  Maverick  missiles  during  a  single 
attack  run.  The  baseline  configuration  consisted  of  a  F-16  cockpit  simulation  using  the 
Low  Altitude  Night  Targeting  and  Infrared  Navigation  (LANTIRN)  targeting  pod  to 
deliver  two  fire-and-forget  Maverick  air-to-surface  missiles.  The  LANTIRN  pod  provides 
a  wide  6°  and  a  narrow  1 .69°  field-of-view  display  through  which  the  pilot  must  detect, 
identify,  and  designate  the  target  vehicles.  Targets  were  KC-10  tankers  parked  among 
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multiple  (2  or  4)  B-52  bombers  on  an  airfield’s  ramps  and  taxiways.  In  a  typical  LAN- 
TIRN  attack,  the  pilot  will  fly  at  medium  altitude  (7500  feet  or  2286  meters)  and  air¬ 
speed  (480  knots  or  247  m/s)  directly  toward  the  target.  The  LANTIRN  targeting  pod 
FUR  display  is  used  to  view  the  target  area  and  to  designate  targets  for  Maverick 
launch.  When  the  target  is  within  the  Maverick’s  optimum  range,  the  pilot  fires  one  or 
more  weapons  at  the  designated  targets. 

The  second  configuration  evaluated  by  the  CCDT  Program  is  an  enhanced 
cockpit  which  was  also  HITL  simulated  and  modeled.  The  enhanced  cockpit  contained 
two  significant  improvements.  First,  the  RTIC  provided  a  satellite  view'  of  the  attack 
area  with  all  targets  marked.  Second,  the  FLIR  was  improved  to  provide  30%  better 
contrast  than  the  baseline  FUR  image.  It  was  anticipated  that  the  advanced  knowledge 
of  the  target  location  coupled  with  the  improved  LANTIRN  FLIR  contrast  would  reduce 
the  target  acquisition  time  and  increase  target  designation  accuracy. 

The  CCDT  Program  had  already  developed  two  other  models  of  the  baseline 
and  enhanced  configurations  which  were  created  to  determine  how  well  the  models 
predicted  the  HITL  simulation  performance.  One  model  was  built  using  the  Extended 
Air  Defense  Simulation  (EADSIM)  while  the  other  was  developed  using  MicroSaint™ 
modeling  software.  The  CCDT  Program  wished  to  compare  the  Soar  model  with  all 
three  of  the  previous  efforts.  Thus,  the  present  modeling  effort  was  informed  only  of  the 
dependent  measures  used  in  the  other  simulations  and  was  otherwise  kept  “in  the 
blind”  to  the  results  of  the  HITL,  EADSIM,  and  MicroSaint™  simulations  so  that  a  fair 
comparison  could  be  made.  The  performance  of  the  other  simulations  was  provided 
only  after  the  Soar  simulation  was  created  and  data  collection  runs  were  complete. 

One  problem  identified  early  on  about  using  Soar  to  model  the  CCDT  Maverick 
launch  mission  is  that  this  particular  mission  is  very  sequential  in  nature.  Hence,  the 
mission  was  not  particularly  mentally  challenging  and  could  not  utilize  the  advanced 
cognitive  modeling  features  of  Soar,  i.e.,  sub-goaling  and  chunking.  The  mission  out¬ 
come  is  largely  based  on  perception  rather  than  cognition  and  calls  for  a  modeling  ap¬ 
proach  more  suited  to  discrete  event  simulations  rather  than  Soar’s  cognitive  approach. 
Hence,  the  bulk  of  the  results  reflect  the  ability  to  predict  the  perceptual  tasks.  Under- 
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standing  this  lack  of  stress  on  Soar’s  capability,  approval  to  build  a  Soar  model  of  the 
Maverick  launch  mission  was  granted.  The  goal  was  to  determine  whether  Soar  could 
be  used  to  micro-model  pilot  behaviors  (STOW  programming  is  at  a  tactical  level),  per¬ 
ceptual  input,  and  motor  output  processes.  This  limited  effort  could  be  followed  by  a 
more  comprehensive  one  which  modified  the  mission  with  more  cognitively-taxing  char¬ 
acteristics  which  would  completely  test  Soar’s  ability  to  cognitively  model  pilots  with  the 
hopes  of  developing  a  cognitive  engineering  tool  for  automatic  testing  of  mentally- 
demanding,  human  computer  interfaces. 

SIMULATION  DESIGN  AND  METHODS 
Summary  of  Requirements 

Mission 

The  ground  attack  mission  begins  with  the  pilot  and  vehicle  at  an  altitude  of  7500 
feet,  flying  at  a  speed  of  480  knots,  and  heading  east  toward  the  first  waypoint,  way- 
point  one.  As  indicated  in  Figure  1,  the  pilot  then  turns  90  degrees  left  and  heads  due 
North  to  the  IP  point  and  then  continues  due  North  to  the  target  area  where  the  pilot  will 
designate,  lock,  and  fire  two  Maverick  missiles  at  two  KC-10  aircraft  on  the  ground 
(there  are  also  four  to  eight  B-52  aircraft  on  the  ground  and  in  the  proximity  of  the  KC- 
10s  which  serve  as  distracters).  The  starting  point  is  at  least  5.1km  from  waypoint  one 
and  waypoint  one  is  5.1km  from  the  IP.  All  reported  times  are  relative  to  the  time  the 
pilot  reaches  the  IP  point.  All  reported  distances  are  relative  to  the  target  or  target 
area. 

The  pilot  knows  there  are  four  possible  plane  groupings  for  each  trial,  but  only 
two  of  the  four  groupings  will  be  populated  in  a  given  trial.  The  pilot  knows  that  in  each 
of  the  populated  groups  there  will  be  only  one  target  aircraft,  a  KC-10.  Hence,  once 
the  pilot  has  identified  and  designated  the  KC-10  in  the  first  group,  the  pilot  can  assume 
all  the  remaining,  yet  to  be  identified  aircraft  for  that  group  are  B-52s.  Thus,  the  pilot 
will  not  identify  the  remaining  aircraft  in  that  group  and  the  pilot  can  switch  its  attention 
to  the  second  group.  If  the  pilot  miss  identifies  the  KC-10  as  a  B-52  and  identifies  all 
the  planes  in  a  grouping  as  B-52s.  then  the  pilot,  knowing  there  must  be  one  KC-10  in 
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the  group,  will  “look”  again  at  each  plane  until  one  plane  is  identified  as  a  KC-10.  After 
the  pilot  designates  the  second  aircraft,  the  pilot  switches  to  locking  each  of  the  Maver¬ 
icks.  The  pilot  then  watches  the  range  icon  until  the  range  is  less  than  58,000  feet. 
When  the  range  is  less  than  58,000  feet,  the  pilot  then  pushes  the  fire  button. 
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Figure  1.  ModSAF  Aerial  View  of  Ground  Attack  Mission. 
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The  behavior  sequence  is  detailed  graphically  in  Figures  2  and  3  and  textually  in 
the  Appendix  1 .3  and  1 .4.  Basically,  there  are  two  separate  scenarios.  The  baseline 
scenario  requires  the  pilot  to  detect  a  cluster  of  objects,  then  visually  detect  and  identify 
each  of  these  objects  until  a  KC-10  is  identified.  After  each  instance  that  the  pilot 
claims  to  detect  a  KC-10,  the  pilot  designates  that  aircraft.  After  the  pilot  designates 
two  aircraft,  the  pilot  then  switches  to  locking  each  of  the  Mavericks.  Finally,  the  pilot 
watches  the  range  icon  and  when  the  pilot  is  within  58,000  feet,  the  pilot  pushes  the  fire 
button. 
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Figure  2.  Flowchart  of  Baseline  Behaviors. 
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Figure  3.  Flowchart  of  RTIC  Behaviors. 


In  the  second  scenario,  the  RTIC  condition,  the  pilot  is  given  a  satellite  view  of 
the  aircraft  on  the  ground  and  the  target  aircraft,  i.e.,  the  KC-10  is  highlighted.  The  so¬ 
lution  to  representing  this  graphical  intelligence  information  was  to  calculate  the  cen¬ 
troid  for  each  group  and  then  to  calculate  the  polar  position  of  each  aircraft  in  the  group 
relative  to  the  centroid.  The  target  was  then  the  aircraft  that  matches  the  relative  angle 
and  distance  to  the  group’s  centroid.  Hence,  after  detecting  all  the  planes  in  the  first 
group,  the  pilot  designates  the  plane  that  matches  this  intelligence  information.  After 
the  pilot  detects  all  of  the  planes  in  the  second  group,  the  pilot  designates  the  plane 
that  matches  that  intelligence  information.  After  the  pilot  has  designated  two  aircraft, 
the  pilot  then  switches  to  locking  each  of  the  Mavericks.  Finally,  the  pilot  watches  the 
range  icon  and  when  the  pilot  was  within  58,000  feet,  the  pilot  pushed  the  fire  button. 
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After  the  pilot  pushed  the  fire  button,  the  program  recorded  the  trial  number,  the 
number  of  hits,  misses,  false  alarms,  and  correct  rejections.  The  program  also  records 
the  time  elapsed  since  the  IP  point  and  the  range  to  the  target  and  range  to  the  pseudo 
airfield  for  the  following  parameters: 

•  First  detection 

•  First  identification  (0  for  the  RTIC  scenario) 

•  First  and  second  designation 

•  First  and  second  Maverick  lock 

•  Launch 


Soar  Behavior  Design 

Visual  Model 

The  existing  Soar  visual  model  had  a  range  of  seven  nautical  miles  with  perfect 
visibility,  i.e.,  at  distance  of  7.001  nautical  miles,  the  Soar  pilot  had  no  information  about 
an  object.  However,  at  6.999  nautical  miles,  the  Soar  pilot  knew  all  it  could  about  the 
object:  lateral  range,  slant  range  altitude,  heading,  speed,  location,  call-sign,  and  vehi¬ 
cle  type.  To  simulate  WFOV  and  NFOV  characteristics,  the  existing  Soar  behaviors 
were  enhanced.  First,  the  Soar  visual  lateral-range  was  increased  to  32,000  meters 
(104,918  feet).  After  an  aircraft  was  detected,  the  object  was  then  assigned  a  detection 
slant-range  and  an  identification  slant-range.  The  detection  range  assigned  was  de¬ 
signed  to  have  an  average  value  of  about  28.6  km  with  a  standard  deviation  of  about 
1.0  km.  The  distance  of  28.6  km  was  chosen  because  at  this  range,  both  the  KC-10 
and  B-52  would  subtend  about  five  pixels  on  a  300x300  pixel  screen  of  a  6.0°  WFOV 
sensor  as  indicated  by  Equation  1  and  Figure  4. 

sinG  =  opp/hyp  or  sin(0.05°)=25m/28.6km  EQ  .1 
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Figure  4.  Diagram  of  WFOV  Angels  and  Pixels  Used  to  Calculate 
Slant-Range  Detection  Limit. 

Determining  the  range  at  which  a  pilot  could  discriminate  between  a  KC-10  and  a 
B-52  required  human  subject  data.  Image  pixelation  was  calculated  based  on  aircraft 
size  and  range.  The  pixel  subtense  is  plotted  in  Figure  5. 


Figure  5.  Estimation  of  an  Aircraft’s  Length  in  Pixels  vs  Range  for  WFOV  and 
NFOV  FLIRs.  The  angles  used  were  6.0  and  1.69  degrees  for  WFOV  and  NFOV 
sensors  respectively.  The  aircraft  were  estimated  to  have  a  163  foot  wingspan. 
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These  calculations  were  the  basis  for  creating  pictures  with  the  Image  Process¬ 
ing  Toolbox  of  Matlab  (Thompson  and  Shure,  1995);  an  example  of  the  output  is  shown 
in  Figure  6.  Human  subjects  were  asked  to  identify  pixelated  reproductions  of  KC-10s 
and  B-52s  viewed  from  15°  angle  above  the  horizon  to  approximate  the  medium  altitude 
approach. 


Figure  6.  One  Recognition  Test  Plate  for  KC-10  Viewed  60“  Off-Axis 

from  the  Nose  View. 

See  Appendix  1.2  for  a  complete  set  of  B-52  and  KC-10  pixelated  images. 

As  seen  in  Figure  7,  D’  did  not  reach  the  90%  level  until  the  plane  subtended  25 
pixels.  The  NFOV  sensor  has  a  1 .69°  field-of-view.  Thus,  the  minimum  identification 
range  can  be  calculated  similarly  to  the  WFOV  calculations  for  detection  range.  The 
identification  range  is  20.3  km  =  25m/sin(1.69°  x25/300/2). 
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Figure  7.  Ability  of  Novice  Human  Subjects  to  Discrimination  Between  KC>10s 

and  B-52s  Based  Upon  Pixel  Density. 

Thus,  in  the  baseline  scenario,  the  Soar  pilot  models  discrimination  between  KC- 
10  and  B-52  aircraft  by  the  ability  to  see  25  pixels  of  the  aircraft’s  length.  However,  an 
image  subtending  25  pixels  did  not  give  the  Soar  model  perfect  vision.  Behaviors  were 
added  so  that  at  a  displayed  image  of  25  pixels  or  slant  range  of  20  km,  the  pilot  had  a 
90%  chance  of  correctly  discriminating  between  a  KC-10  and  a  B-52.  To  model  human 
behavior,  the  identification  accuracy  was  dependent  upon  distance  according  to  the 
following  equations 


PidentifI cation  ~  ^ 

Slant  Range  >  96,000  feet 

EQ.2 

P identification  "  2.93-3.045E-5  range 

96,000  <  Slant  Range  >  63,000  feet 

EQ.3 

^identification  ““  ^ 

Slant  Range  >  63,000  feet 

EQ.4 

As  seen  in  Figures  8  and  9,  identification  distance  had  an  element  of  chance 

associ- 

ated  with  both  the  RTIC  and  baseline  behaviors. 
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Number  of  Occurrences 


First  Detection  Distribution 


Figure  8.  First  Detection  Distribution  for  RTIC  Condition. 


Figure  9.  First  Identification  Distribution  for  the  Baseline  Condition. 
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In  the  RTIC  scenario,  the  pilot  did  not  need  to  visually  discriminate  between  the 
KC-10  and  the  B-52.  The  pilot  only  needed  to  determine  where  the  group  of  planes 
were,  match  the  NFOV  sensor  image  of  the  planes  to  the  intelligence  image,  and  then 
recognize  the  relative  position  of  the  target  plane  relative  to  the  other  planes  in  the 
group  and  then  designate  this  same  plane  in  the  NFOV  sensor  image. 

The  solution  to  representing  this  graphical  intelligence  information  was  to  calcu¬ 
late  the  centroid  for  each  group  of  aircraft.  Then,  the  polar  position  of  each  aircraft 
relative  to  the  centroid  was  calculated.  The  target  was  then  the  aircraft  that  was  both 
within  20  meters  and  0.1  radians  of  the  calculated  distance  between  the  centroid  and 
KC-10  given  in  the  intelligence  information.  Since  the  target  position  changed  twice 
and  the  number  of  distracters  changed  twice,  this  caused  the  location  of  the  centroid  to 
be  changed  four  times  which  required  four  sets  of  intelligence  information.  This  map¬ 
ping  between  the  Soar  supplied  data  located  in  files  rl.soar,  r2.soar,  rS.soar  and 
r4.soar  and  the  plane  layout  configurations  are  seen  in  Table  1. 


Table  1.  The  Associative  Mapping  Between  Soar  Supplied  RTIC  Information 
and  Plane  Layout  Sourced  at  the  ModSAF  Parser 


FILE 

NAME 

PLANES 
PER  GROUP 

TARGET 

POSITION 

PLANE  LAYOUT 

(Corresponding  file  sourced  at  ModSAF  parser) 

r1  .soar 

3 

1 

1  -  6 

rZ.soar 

3 

2 

7-12 

rS.soar 

5 

1 

13-18 

r4.soar 

5 

2 

19-24 

Motor  Model 

Since  the  Soar  language  does  not  have  any  inherent  mechanism  for  generating 
delays,  the  top-state  attribute  current-time  was  used  as  a  timer.  The  resolution  of  the 
current-time  attribute  was  one  second.  Hence,  a  delay  set  for  one  second  has  a  theo¬ 
retical  delay  of  greater  than  zero  but  less  than  or  equal  to  one  second.  Behavioral  mo¬ 
tor  delays  were  added  to  prevent  immediate  execution  of  behaviors  where  motor  delays 
were  required.  A  summary  of  motor  delays  and  the  value  of  the  programmed  delays 
are  seen  in  Table  2. 
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Table  2.  Summary  of  Simulated  Motor  Activity  and 
Corresponding  Programmed  Delay 


MOTOR  ACTIVITY 

PROGRAMMED  MOTOR  DELAY 

Engage  WFOV 

1 

Switch  to  NFOV 

1 

Decide  Hit,  Miss,  FA  or  OR 

1 

Designation  and  Confirm 

1 

Switch  from  Designation  to  Locking 

3 

Locking  and  Confirm 

1 

Launch 

2 

Randomization 

Since  Soar  does  not  have  a  random  number  generator,  random  numbers  were 
generated  by  caiculating  the  sine  vaiue  for  the  current-time  vaiue.  To  obtain  a  distribu¬ 
tion  that  was  not  flat  and  had  similarities  to  a  gaussian  distribution,  three  random  num¬ 
bers  were  averaged.  To  calculate  the  detection  and  identification  ranges  for  each 
plane,  the  average  of  three  random  numbers  were  subtracted  from  0.5  and  this  differ¬ 
ence  was  multiplied  by  the  variance  for  that  parameter."  This  result  was  then  added  to 
the  expected  mean  for  that  parameter.  As  seen  in  Figure  8  and  9,  the  distribution  of 
detection  and  identification  ranges  have  a  gaussian-like  distribution. 

After  the  Soar  pilot  is  focused  on  a  particular  plane  and  the  slant  range  is  less 
than  the  calculated  identification  range  for  that  plane,  then  the  probability  for  accurately 
identifying  the  focused  vehicle  is  calculated  as  seen  in  equations  2,  3,  and  4.  This  cal¬ 
culation  is  based  upon  the  slant-range  between  the  Soar  pilot  and  the  aircraft  in  ques¬ 
tion  and  a  randomly-generated  number  having  a  flat  distribution.  If  the  calculated  prob¬ 
ability  was  greater  than  the  randomly-generated  number,  then  a  hit  or  correct  rejection 
was  recorded.  If  the  calculated  probability  was  less  than  the  randomly-generated  num¬ 
ber,  then  a  miss  or  false  alarm  was  recorded. 

Program  Description 

As  seen  in  Figures  2  and  3  (Flowcharts  for  Baseline  and  RTIC  Behaviors)  and 
Appendices  1 .3  and  1 .4  (selected  text  outputs  from  execution  of  baseline  and  RTIC  be¬ 
haviors),  the  mission  starts  with  the  Soar  pilot  flying  toward  waypoint  one.  After 
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achieving  waypoint  one,  the  Soar  pilot  turns  north  and  flies  to  the  IP.  After  achieving 
the  IP  waypoint,  a  timer  is  started,  the  pilot  continues  North  at  an  altitude  of  7,500  feet 
and  speed  of  480  knots  and  engages  the  WFOV.  After  a  cluster  of  objects  is  detected, 
the  Soar  pilot  switches  to  NFOV.  The  pilot  continues  to  focus  on  one  object  in  the 
cluster  until  it  is  both  detected  as  a  plane  and  identified.  After  identification,  the  validity 
of  the  identification  is  calculated  and  recorded  as  a  hit,  miss,  false  alarm,  or  correct  re¬ 
jection.  Once  a  KC-10  is  believed  to  be  identified,  a  hit  or  false  alarm,  then  the  object  is 
designated  and  the  second  cluster  is  processed.  If  a  miss  occurs,  then  all  of  the  identi¬ 
fication  information  for  that  group  is  erased  after  all  the  planes  are  identified  and  the 
identification  process  is  repeated  until  a  hit  or  false  alarm  occurs.  After  two  aircraft  are 
designated,  the  pilot  switches  to  the  launch  procedures.  The  launch  procedures  consist 
of  locking  and  confirming  both  of  the  Mavericks.  After  locking  both  Mavericks,  if  the 
slant  range  is  greater  than  58,000  feet,  then  the  pilot  watches  the  range  icon.  After 
locking  both  Mavericks,  if  the  slant  range  is  less  than  58,000  feet,  the  pilot  presses  the 
fire  button. 

The  RTIC  behavior  scenario  differs  from  the  baseline  behavior  outlined  above  in 
the  identification  process.  In  the  RTIC  behaviors  the  planes  are  not  identified.  After  the 
cluster  is  detected  in  WFOV  and  the  pilot  switches  to  NFOV,  the  pilot  focuses  on  each 
target  and  calculates  the  polar  coordinates  for  that  plane.  If  the  plane’s  polar  coordi¬ 
nates  are  not  within  the  limits  input  by  the  intelligence  information,  then  a  correct  rejec¬ 
tion  is  recorded  and  the  pilot  proceeds  to  focus  on  the  next  plane  In  the  group. 

If  the  plane’s  polar  coordinates  are  within  the  limits  input  by  the  intelligence  in¬ 
formation,  then  a  hit  is  recorded,  the  plane  Is  designated  and  the  pilot  proceeds  to  the 
next  group  or  to  launch  procedures  if  this  was  the  second  group. 

RESULTS 

Analysis  of  Soar  Model 

The  experiment  contained  two  major  variables,  scene  density  and  pilot  vehicle 
interface.  The  scene  was  further  complicated  in  that  only  two  of  the  four  possible  vehi¬ 
cle  locations  were  occupied.  A  bird’s  eye  view  of  the  field  arrangement  is  presented  in 
Figure  10.  This  requirement  resulted  in  six  different  possible  groupings  (1-2, 1-3, 1-4, 
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2-3,  2-4,  3-4).  Furthermore,  the  location  of  the  target  vehicle,  the  KC-10,  was  also  var¬ 
ied  between  two  locations  within  each  group.  Hence,  the  experimental  matrix  was  a 
2x2x6x2  which  was  repeated  five  times  for  a  total  of  240  trials.  Table  3  summarizes 
these  independent  measures. 


Figure  10.  Airfield  Area  (from  Holly  Bautsch,  1997  pg.  27). 
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Table  3.  Arrangement  of  Independent  Measures  Used 
in  Prior  Efforts  and  by  the  Soar  Model 


Variable 

Levels 

Variations 

Pilot  Vehicle  Interface 

2 

Baseline 

Enhanced  (RTIC) 

Cluster  Size 

2 

Three  aircraft 

Five  aircraft 

Groupings 

6 

Two  out  of  four  possible  airfield  positions 

1-2, 1-3, 1-4, 2-3, 2-4, 3-4 

Position  in  Cluster 

2 

Two  positions  used  each  in  groups  size  three  or  five 

The  dependent  variables  measured  included:  detection  time  and  range,  identifi¬ 
cation  time  and  range,  designation  time  and  range,  lock  time  and  range,  launch  time 
and  range,  and  recognition  accuracy.  As  seen  in  Figures  8  and  9,  the  modeling  of  de¬ 
tection  and  identification  ranges  did  have  a  distribution  that  appears  gaussian.  The  in¬ 
tended  detection  range  average  was  93,500  feet  with  a  standard  deviation  of  about 
3,200  feet.  The  actual  detection  slant-range  for  all  trials  was  93,989  with  a  standard 
deviation  of  3,247  feet.  Similarly,  the  intended  identification  range  average  was  66,500 
feet  with  a  standard  deviation  of  about  1 ,600  feet.  The  actual  identification  slant-range 
for  all  trials  was  67,030  with  a  standard  deviation  of  1 ,729  feet. 

As  expected  from  analysis  of  the  visual  model,  the  Soar-simulated  RTIC  inter¬ 
face  resulted  in  significantly  earlier  designation  and  lock  times  compared  to  the  baseline 
condition  (first  designation  time:  p=0.001,  F=3352,  N=60;  first  lock  time:  p=0.0001, 
F=3872,  N=60).  As  seen  in  Figure  15,  the  first  lock  time  for  the  RTIC  condition  with  a 
scene  complexity  of  three  averaged  only  21 .25  seconds.  The  RTIC  condition  with  a 
scene  complexity  of  five  was  2.2  seconds  higher,  due  to  the  increased  scene  complex¬ 
ity.  Conversely,  the  baseline  condition  with  a  scene  complexity  of  three  had  an  aver¬ 
aged  first  lock  time  of  54.72  seconds,  33.  47  seconds  longer  than  the  RTIC  condition. 
Similarly,  scene  complexity  increased  baseline  first  lock  time  by  1.58  seconds.  Al¬ 
though  the  PVI  and  scene  complexity  interaction  was  not  statistically  significant,  the  in¬ 
creased  effect  for  the  RTIC  condition  should  have  resulted  because  in  the  RTIC  condi- 
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tion,  the  Soar  pilot  must  process  each  of  the  planes  prior  to  identifying  the  target.  How¬ 
ever,  in  the  baseline  condition,  the  Soar  pilot  can  terminate  processing  a  group  after  the 
KC-10  is  identified.  Thus,  in  the  baseline  condition,  the  average  number  of  vehicles 
expected  to  be  processed  with  a  scene  complexity  of  three  is  two  and  with  a  scene 
complexity  of  five  is  three.  Hence,  increasing  the  scene  complexity  in  the  baseline 
condition  will  not  increase  the  average  lock  time  as  much  as  the  RTIC  condition,  but  the 
variation  should  be  much  larger  in  the  baseline  condition.  The  variation  in  RTIC  first 
lock  times  are  about  4.5  seconds  regardless  of  scene  complexity.  However,  baseline 
lock  time  variation  is  3.37  seconds  for  a  scene  complexity  of  three  and  increases  to 
4.04  seconds  for  a  scene  complexity  of  five.  Although  this  time  difference  in  variance  is 
not  significant  (F=1 .44,  N=60),  the  difference  in  variance  for  first  lock  range  for  baseline 
condition  between  scene  complexities  is  significant  (F=1.88,  N=60).  The  explanation 
for  the  significance  is  that  the  time  has  an  increased  variance  due  to  the  different  times 
it  takes  to  reach  the  same  slant  range  from  each  group  since  the  target  groups  are  lo¬ 
cated  at  different  distances  from  the  IP. 

Overall  accuracy  of  performance  was  comparable  for  the  four  experimental 
conditions.  Categories  of  responses  for  each  condition  are  plotted  in  Figure  1 1 .  The 
most  marked  difference  is  the  increased  number  of  correct  rejections  when  more  B-52s 
appeared  in  target  clusters. 
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Figure  1 1 .  Summary  of  Soar  Simulation  Times  for  Different  Scene 
Complexities  and  Different  PVIs. 


Soar  Simulation  Measure  of  Performance 


Figure  12.  Summary  of  Soar  Simulation  Performance  Measurements 
for  Different  Scene  Complexities  and  Different  PVIs. 


The  launch  times  between  all  conditions  were  very  similar.  The  baseline  behav¬ 
iors  typically  resulted  in  the  identification  of  targets  and  the  locking  of  the  Maverick 
missiles  inside  the  maximum  launch  range.  Hence,  the  baseline  Soar  pilot  was  typically 
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not  waiting  for  the  range  icon  to  reach  58,000  feet.  This  resulted  in  a  launch  variation 
that  was  less  than  first  lock  variation  due  to  clipping  at  the  maximum  launch  range  tail  of 
the  distribution  as  seen  in  Figure  12.  Conversely,  the  variation  in  RTIC  launch  ranges 
was  very  small  as  seen  in  Figure  13  with  range  scales  comparable  to  the  baseline 
condition.  This  greatly  reduced  variation  because  in  all  the  RTIC  conditions,  the  Soar 
pilot  was  always  watching  the  range  icon  waiting  to  pass  the  maximum  firing  range  of 
58,000  feet. 


Figure  13.  Distribution  of  Launch  Ranges  for  Soar  Model 
of  Baseline  Configuration. 
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RTIC  Launch  Distribution 


Figure14.  Distribution  of  Launch  Ranges  for  Soar  Model 
of  RTIC  Configuration. 


Comparison  of  Soar  Simulation  Results  to  HITL,  TTA,  and  CTA  Simulations 

After  completion  of  the  Soar  simulation  data  collection  process,  data  from  HITL 
testing  and  other  simulation  models  were  compared  to  the  Soar-simulated  results.  The 
results  from  HITL,  Soar,  Traditional  Task  Analytic  (TTA),  and  Cognitive  Task  Ana¬ 
lytic  (CTA)  are  summarized  in  Table  4.  As  seen  in  Figure  15,  the  Soar,  TTA,  and  CTA 
simulations  grossly  under-estimated  the  first  lock  time  for  the  baseline  conditions,  but 
Soar  did  surpass  the  TTA  and  CTA  models  in  predicting  RTIC  first  lock  times.  The 
Soar  simulation  over-estimated  the  first  lock  time  in  the  sparser  scene  complexity  indi¬ 
cating  an  under-estimation  of  the  detection  range  in  the  visual  model  for  the  RTIC 
condition.  In  fact,  close  examination  of  the  HITL  data  indicated  that  some  pilots  were 
locking  targets  prior  to  the  IP  point.  Furthermore,  Soar  was  similar  to  the  other  models 
in  accurately  indicating  that  additional  distracters  would  increase  the  time  required  for  a 
pilot  to  lock  the  missiles;  although  all  simulations  under-estimated  the  HITL  data. 
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First  Lock  Times 


Baseline-3  Baseline-5  RTIC-3  RTlC-5 

Condition 


Figure  15.  Comparison  of  HITL  First  Lock  Times  to  Soar,  TTA, 
and  CTA  Base  Simulations. 
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Table  4.  Data  Summary  of  Soar  HITL,  TTA,  and  CTA  Data 
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Soar  estimated  the  second  lock  time  much  better  than  the  first  as  indicated  in 
Figure  16.  However,  this  improved  estimate  was  due  to  the  cancellation  of  a  sluggish 
visual  model  and  a  hyperactive  motor  activity  model.  Given  those  limitations,  one 
would  expect  that  those  differences  would  be  canceled  out  when  comparing  PVIs  of  the 
same  scene  density.  The  HITL  data  indicated  that  for  a  scene  complexity  of  three,  the 
RTIC  condition  reduced  the  average  second  lock  time  by  17.78  seconds  as  compared 
to  Soar,  TTA,  and  CTA  which  reduced  the  averaged  second  lock  time  by  33.48,  26.75, 
and  14.76  respectively.  This  data  is  summarized  in  Table  5. 

Careful  analysis  of  the  available  HITL  data  shows  an  apparent  confound.  The 
presentation  order  to  the  human  pilots  appears  not  to  be  counter-balanced  for  presen¬ 
tation  order  with  respect  to  scene  complexity.  Apparently,  for  a  given  PVI,  a  scene 
complexity  of  five  was  always  given  first  and  a  scene  complexity  of  three  seconds.  The 
HITL  data  for  each  subject  was  taken  in  four  sets  of  four  trials  where  the  baseline  and 
RTIC  conditions  were  interwoven.  The  difference  between  sets  for  a  given  PVI  Is  seen 
in  Figure  17  as  a  step  function  between  the  fourth  and  fifth  trials  which  corresponds  to 
the  end  of  the  first  set  and  beginning  of  the  second  set.  Whether  this  difference  is  due 
to  a  learning  effect  in  the  RTIC  condition  or  a  training  effect  is  confounded. 
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Figure  16.  Comparison  of  HITL  Second  Lock  Times  to  Soar,  TTA, 
and  CTA  Base  Simulations. 
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Table  5.  Differential  Comparison  of  Second  Lock  Times. 

The  scene  density  was  three  for  both  PVI  conditions  (all  times  are  relative  to  the  IP). 


Baseline-3 

(sec.) 

RTIC-3 

(sec.) 

Difference 

(sec.) 

Error 

(%) 

HITL 

44.7 

26.92 

17.78 

SOAR 

55.73 

22.25 

33.48 

88 

TTA 

61.94 

35.19 

26.75 

50 

CTA 

55.57 

40.81 

14.76 

-17 

The  second  lock  times  also  reveal  a  cognitive  effect  in  the  comparison  of  in¬ 
creased  scene  complexity.  In  the  baseline  condition  with  three  vehicles,  on  average, 
the  pilot  will  have  to  identify  two  aircraft  before  finding  the  KC-10.  In  the  baseline  con¬ 
dition  with  five  aircraft,  on  average  the  pilot  will  have  to  identify  three  aircraft  before 
finding  the  KC-10.  Thus,  in  the  baseline  condition,  typically  the  pilot  only  needs  to  fo¬ 
cus  upon  one  additional  vehicle,  whereas  the  RTIC  condition  requires  focusing  on  two 
additional  aircraft  when  the  scene  complexity  is  increased  from  three  to  five.  Hence, 
scene  complexity  is  expected  to  increase  second  lock  times  for  the  RTIC  condition  in 
comparison  to  the  baseline  condition.  As  seen  in  Table  6,  both  HITL  data  and  Soar 
simulation  data  have  increased  scene  complexity  differences  for  the  RTIC  condition  as 
compared  to  baseline.  However,  the  TTA  and  CTA  models  poorly  simulate  the  RTIC 
scene  complexity  differences. 

Table  6.  Scene  Complexity  Average  Differences  for  Second  Lock  Times. 


Model 

Baseline 

(sec.) 

RTIC 

(sec.) 

Change 

(%) 

HITL 

4.67 

13.01 

179 

SOAR 

1.57 

2.3 

47 

TTA 

0.42 

-0.02 

-105 

CTA 

0.42 

-0.10 

-124 
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Analysis  of  launch  time  differences  between  conditions  is  difficult.  In  the  base¬ 
line  conditions,  the  HiTL  and  Soar  data  are  very  similar.  This  increased  similarity  over 
second  lock  time  is  attributed  to  an  apparently  hyper  motor  model  of  the  Soar  simula¬ 
tion.  What  is  iost  in  the  Soar  simulation  is  a  difference  between  PVI  conditions.  This  is 
because  the  Soar  model  used  a  maximum  launch  range  of  58,000  feet.  Since  the 
baseline  behaviors  were  near  completion  at  this  point  and  RTIC  behaviors  aiways 
waited  to  pass  this  threshold,  the  benefits  of  the  RTIC  behaviors  are  not  identifiable  for 
this  measurement.  Conversely,  HITL  data  show  decreased  RTIC  launch  times  because 
that  data  did  not  appear  to  have  a  maximum  Maverick  launch  range  (one  pilot  launched 
from  76,650  feet). 

Analysis  of  launch  range  is  similar  to  launch  time  except  the  TTA  and  CTA  data 
are  problematic.  As  seen  in  Figures  18  and  19,  it  is  unknown  how  launch  times  for 
HITL,  TTA,  and  CTA  can  be  so  similar,  but  launch  ranges  can  be  drastically  different. 


Figure  18.  Comparison  of  HITL  Launch  Times  to  Soar,  TTA, 
and  CTA  Based  Simulations. 
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NOTE:  The  better  results  are  the  launch  times  with  the  lower 
values  because  these  launches  occur  closer  to  the  IP 
point  and,  therefore,  further  from  the  target  area. 
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Figure  19.  Comparison  of  HITL  Launch  Ranges  to  Soar, 

TTA,  and  CTA  Based  Simulations. 

Table  7  shows  the  number  of  false  alarms  increased  in  the  baseline  condition  as 
scene  complexity  increased  for  HITL,  Soar,  and  TTA  simulations.  However,  the  num¬ 
ber  of  misses  decreased  as  scene  complexity  increased.  The  explanation  is  that  as  the 
number  of  distracters  increase,  the  number  of  false  alarms  will  increase,  but  the  number 
of  targets  remained  constant  so  the  number  of  misses  should  be  constant.  However, 
the  experimental  procedure  was  to  ignore  the  remaining  vehicles  once  a  target  was 
identified.  Hence,  the  increase  in  false  alarms  will  decrease  the  number  of  opportuni¬ 
ties  to  classify  the  KC-10s  and  thus  decrease  both  the  opportunity  for  and  number  of 
misses. 
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Table  7.  Performance  Measurements. 

HITL  responses  are  compared  to  Soar,  TTA,  and  CTA  simulation  responses. 


Number  of  False  Alarms 

Number  of  misses 

Number 

of 

Trials 

Condition 

Baseline 

RTIC 

Baseline 

RTIC 

3 

5 

3 

5 
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5  1 

PITL ' . 
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0 

0 

0 

0 

Bl 

6 
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DISCUSSION 

The  Soar  model  of  the  Maverick  strike  mission  was  limited  by  that  mission  sce¬ 
nario  in  that  it  forced  the  model  to  overemphasize  the  perceptual  aspect  of  human  cog¬ 
nitive  performance.  The  model  did  demonstrate  Soar’s  ability  to  perform  discrete  event 
simulations,  but  the  model  did  not  meaningfully  employ  Soar’s  ability  to  represent  the 
more  sophisticated  aspects  of  human  cognitive  processing.  For  this  reason,  this  dis¬ 
cussion  will  focus  less  on  the  present  model’s  capabilities  and  look  forward  to  future 
application  of  Soar  to  cognitive  modeling.  The  discussion  will  first  identify  classes  of 
events  which  cognitive  models  must  accurately  predict.  Second,  experiments  which 
utilize  the  cognitive  aspect  of  the  Soar  architecture  for  modeling  human  cognition,  in¬ 
cluding  human  limitations  and  human  errors,  will  be  identified.  Third,  we  shall  discuss 

s 

how  to  compare  competing  human  performance  models.  Finally,  how  to  compare  a 
cognitive  model’s  predictions  with  human  performance  will  be  discussed. 

Choosing  Tasks  Suitable  for  Cognitive  Modeling 

The  Maverick  mission  biased  the  model  toward  the  perception  process  since  the 
task  required  the  pilot  to  pick  a  vehicle  to  strike  based  on  various  sensory  resources 
provided  in  the  two  cockpit  configurations  being  tested.  The  mission  characteristics 
drove  the  model  toward  a  perceptual  detection-identification  cycle  dependent  on  the 
range-to-target  and  the  cockpit  information  display  characteristics.  The  augmented 
configuration  cockpit  with  overhead  imagery  eliminated  the  recognition  part  of  the  pilot's 
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task,  making  earlier  firing  decisions  possible.  The  attack  ranges  were  determined  by 
the  ability  to  detect  individual  targets  in  the  target  clusters.  Although  perception  is  part 
of  human  cognition,  it  is  perhaps  the  best  understood  and  the  least  demanding  aspect 
of  cognitive  modeling.  According  to  the  Soar  information  processing  model,  perceptual 
information  is  fed  into  short-term  memory  store  where  it  is  processed  in  combination 
with  long-term  memory  to  enable  the  human  operator  to  make  decisions  and  take  ac¬ 
tion  to  accomplish  the  required  goals.  The  pilot’s  post-perception  cognitive  processing 
for  the  Maverick  mission  was  trivial  (since  target  vehicle  recognition  was  achieved  when 
enough  pixels  were  available  to  support  pilot  perception  of  the  targets)  because  there 
were  no  demanding  additional  cognitive  processes  to  launching  the  Maverick  missiles. 

Tasks  suitable  for  development  and  testing  of  cognitive  models  must  exercise 
most  if  not  all  aspects  of  cognitive  processing,  including  perception,  assimilation  into 
short-term  memory,  consolidation  with  long-term  storage  Information,  decision  proc¬ 
esses,  and  the  motor  response.  Thus,  choice  of  the  task  to  be  modeled  is  absolutely 
critical  to  developing  and  testing  the  modeling  process.  Test  tasks  should  also  permit  a 
variety  of  alternative  operator  actions.  Particular  actions  may  “mark”  the  presence  of 
specific  cognitive  response  generation  processes.  Additionally,  post-performance  de¬ 
briefings  may  reveal  more  information  about  the  cognitive  processes  underlying  subject 
action.  Thorough  test  tasks  can  be  developed  in  two  ways:  a  single  full-task  which 
thoroughly  tests  all  aspects  of  cognition  or  part-tasks  which  test  specific  separate  as¬ 
pects  of  cognition. 

Consider  one  thorough  full-task  based  on  the  existing  Maverick  strike  mission 
that  would  span  all  the  levels  of  cognitive  processing.  This  task  would  require  percep¬ 
tion  of  critical  information,  integrating  the  new  information  with  existing  knowledge,  us¬ 
ing  the  combination  to  make  a  decision,  and  the  resulting  physical  response.  First, 
modify  scenarios  such  that  high  value  targets  (KC-10s)  are  not  present  at  every  attack; 
hence,  the  pilot  needs  to  decide  whether  to  break-off  the  attack.  This  would  modify  the 
detect-recognize  loop  to  include  a  decision  whether  or  not  to  fire.  The  scenario  can  be 
further  elaborated  by  locating  the  target  not  just  among  war  planes  where  an  error  has 
little  consequence,  but  civilian  transports  that  may  or  may  not  contain  passengers.  Now 
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the  detect-recognize-decision  loop  requires  greater  perceptual  precision  (capture  of 
more  details  about  the  aircraft),  additional  information  from  long  term  storage  (a  wider 
range  of  aircraft  types),  and  the  fire  decision  must  consider  collateral  damage. 

The  part-task  alternative  approach  would  consist  of  a  series  of  unrelated  tasks, 
each  testing  a  different  aspect  of  cognition  process  separately.  This  approach  allows 
dissection  of  the  cognitive  process,  but  sacrifices  the  opportunity  to  study  complex  in¬ 
teractions  among  the  levels.  In  this  fashion,  the  Maverick  mission  which  tests  percep¬ 
tual  modeling  would  be  augmented  with  events  that  would  separately  test  other  aspects 
of  cognition.  During  egress  from  the  target  area  a  pop-up  threat  may  appear  that  re¬ 
quires  a  decision  about  how  to  avoid  it.  A  system  emergency  may  occur  which  requires 
weapon  system  knowledge  and  integration  of  prior  knowledge. 

Multiple  decisions  are  another  way  in  which  the  cognitive  complexity  of  the  mod¬ 
eling  test  task  can  be  increased.  When  operators  must  trade-off  outcomes  in  two  or 
more  processes,  as  often  is  the  case  in  real  life  tasks,  the  cognitive  model  can  be 
tested  under  more  realistic  circumstances.  This  can  be  illustrated  in  our  Maverick  mis¬ 
sion  by  the  addition  of  terminal  defenses  in  the  target  area.  The  original  mission  had 
the  pilot  fly  unopposed  into  the  target  area.  The  presence  of  terminal  defenses  would 
add  increasing  risk  with  greater  proximity  to  the  targets.  The  range  of  this  risk  would 
depend  on  the  aircraft  threat  warning  system,  pilot  understanding  of  threat  characteris¬ 
tics  and  mission  priority.  Parallel  processing  of  the  threat  problem  further  complicates 
the  cognitive  processing  demands  on  the  pilot  or  model  and  puts  greater  load  in  the 
latter  stages  (information,  integration  and  decision)  of  the  cognitive  process. 

If  cognitive  modeling  intends  to  predict  human  operator  performance,  models 
must  respond  to  environmental  stressors  like  their  human  counterparts.  Models  must 
be  able  to  incorporate  the  effects  of  temporal  pacing,  effort,  environmental  stress  (tem¬ 
perature,  g-load,  etc.),  and  fatigue.  Combat  environments  are  very  different  from  the 
laboratory  conditions,  so  that  generalization  is  sometimes  unjustified.  Creating  realistic 
models  will  require  collecting  data  under  as  realistic  conditions  as  possible. 
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Enhancing  Soar’s  Similarity  to  Human  Cognition 

Newell  (1987)  intended  Soar  to  perform  cognition  in  a  way  similar  to  that  of  hu¬ 
man  cognition.  However,  the  existing  Soar  architecture  only  partially  achieves  this  goal. 
Although,  in  principle-Soar  functions  in  a  human-like  fashion,  its  computational  repre¬ 
sentations  of  working  and  long-term  memory  are  far  more  powerful  than  their  human 
counterparts.  Soar’s  departures  from  human  cognition  are  well  understood.  Working 
memory  is  meant  to  function  like  short-term  memory,  but  working  memory  has  unlimited 
capacity  while  human  short-term  memory  has  considerably  less  capacity  (Miller,  1957). 
Soar’s  knowledge  base  is  permanent,  monolithic,  and  represented  as  rules  and  their 
firing  conditions  belying  its  artificial  intelligence  production  system  origins.  Human 
knowledge  is  much  more  complex,  marked  by  imperfect  retrieval,  representations 
varying  over  time,  interactions  with  sensory  modalities,  and  configured  into  non-uniform 
schemata  (Neisser,  1976).  Soar  must  be  further  anthropomorphized  for  it  to  accurately 
model  human  cognitive  performance. 

Dr.  Laird  at  University  of  Michigan  (personal  communication),  a  project  consult¬ 
ant,  has  identified  several  ways  that  the  Soar  architecture  can  be  exploited  to  make  its 
information  processing  more  anthropomorphic.  The  changes  Laird  suggests  do  not  al¬ 
ter  Soar’s  basic  function.  The  changes  modify  the  contents  of  working  memory  or 
change  rule  processing  to  mimic  known  human  processing  characteristics. 

The  first  proposed  change  is  meant  to  mimic  high  stress  or  workload  conditions. 
Human  error  rates  increase  under  such  conditions  while  unmodified  Soar  processing  is 
unaffected.  The  change  involves  the  withdrawal  of  rules  from  working  memory  to  mimic 
changes  in  short-term  memory  under  stress.  Those  withdrawn  rules  would  not  fire  and 
the  resulting  decisions  would  not  be  based  on  all  the  available  information.  A  critical 
issue  for  withdrawing  rules  is  choosing  which  to  remove.  Random  withdraw  is  one  way. 
Another  way  is  to  track  the  frequency  of  past  firings  and  lose  less  frequently  firing  rules 
first. 

Second,  Laird  proposes  limiting  the  number  of  productions  fired  in  parallel  under 
stress  or  workload.  This  would  reduce  the  throughput  of  cognitive  processing  and 
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make  cognition  more  of  a  sequential  than  a  parallel  process.  Human  ability  to  perform 
dual  tasks  simultaneously  is  limited  and  affected  by  the  nature  of  the  tasks  (Wickens, 
Sandry  &  Vidulich,  1993). 

A  third  method  for  anthropomorphizing  Soar  is  to  limit  the  cascades  of  production 
firing  during  the  elaboration/decision  cycle.  The  elaboration/decision  cycle  now  consid¬ 
ers  the  consequences  of  every  potential  firing  on  every  other  firing  before  making  deci¬ 
sions.  Human  decision  makers  are  typically  not  as  thorough  in  testing  the  conse¬ 
quences  of  each  decision  and  consequently  make  errors  based  on  the  failure  to  predict 
the  broader  consequences  of  their  decisions.  Limiting  cascades  of  firings  will  increase 
Soar  errors,  but  it  remains  to  be  seen  whether  these  errors  mimic  those  made  by  hu¬ 
man  operators. 

The  next  two  alterations  of  Soar  function  deal  as  much  with  the  supporting  com¬ 
putational  environment  as  they  do  with  Soar  itself.  Laird  suggests  artificially  restricting 
execution  speed,  so  as  to  prevent  Soar  from  thoroughly  processing  the  inputs  in  the 
available  time.  Pilots  in  the  HITL  simulation  had  only  moments  to  detect  and  recognize 
the  strike  targets;  under  less  time  pressure  they  probably  could  have  performed  better. 
Soar,  running  on  a  fast  computer  might  be  able  to  process  more  information  in  the  allot¬ 
ted  time  compared  to  its  human  counterpart.  The  IBM  chess  program  Deep  Blue  beat 
a  human  competitor  by  thoroughly  searching  alternative  courses  of  action  faster  and 
more  accurately  than  its  human  opponent. 

Another  way  processing  speed  can  be  limited  is  by  adding  additional  tasks  to  re¬ 
duce  the  processing  time  available  for  the  cognitive  processing.  Pilots  devote  signifi¬ 
cant  mental  effort  to  flying  their  aircraft,  a  process  that  competes  with  available  re¬ 
sources  to  detect,  identify,  and  strike  their  targets.  This  siphoning  of  cognitive  capacity 
is  managed  in  two  different  ways  in  the  design  of  strike  aircraft.  One  design  strategy  is 
to  include  a  second  crew  person  dedicated  to  operating  weapons.  The  F-15E  Strike 
Eagle  has  a  two  person  crew,  one  of  whom  is  a  dedicated  weapon  systems  officer. 
The  other  design  strategy  is  to  automate  some  of  the  tasks  so  that  a  single  crew  person 
can  perform  the  whole  mission.  The  F-117  Nighthawk  opts  for  the  latter  method,  auto¬ 
mating  flying  while  the  pilot  concentrates  on  operating  the  weapon  systems.  Loading 
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the  processor  with  ancillary  tasks  to  reduce  Soar  processing  cycle  times  mimics  the 
workload  issue  in  humans.  Furthermore,  Soar’s  processing  speed  must  be  matched  to 
the  amount  of  information  a  human  operator  can  consider  per  the  available  processing 
time.  Either  the  restriction  of  execution  speed  or  the  addition  of  competing  tasks  will 
require  empirical  studies  to  determine  how  much  elaboration  cycle  time  for  any  given 
Soar-hardware  implementation  are  comparable  to  human  performance  and  how  elabo¬ 
ration  cycle  time  may  vary  with  training  and  experience. 

The  last  potential  modification  to  Soar  to  increase  its  similarity  to  human  cogni¬ 
tion  is  to  vary  the  design  of  its  knowledge  base.  The  form,  organization,  and  order  of 
the  rules  with  which  Soar  operates  on  its  perception  is  central  to  cognitive  processing. 
Neisser  (1976)  discusses  the  role  of  schemata  or  the  organization  of  knowledge  in  the 
cognitive  process.  Hutchins  (1995)  further  shows  how  very  different  schemata  can  be 
employed  to  support  the  same  behavior,  in  his  case,  vessel  navigation.  In  a  simple 
form,  novice  behaviors  can  be  compared  to  expert  behaviors  based  on  the  depth  and 
organization  of  their  knowledge  bases.  However,  more  subtle  differences  in  the  knowl¬ 
edge  base  may  provide  greater  understandings  of  Soar  model  behavior  and  ultimately 
human  behavior. 

Making  Comparisons  Among  Competing  Cognitive  Models 

This  was  one  aspect  of  the  current  modeling  effort  that  was  very  well  managed 
by  the  Air  Force  Research  Laboratory.  Both  the  HITL  and  EADSIM  data  collections 
preceded  development  of  the  Soar  model,  so  our  Soar  development  team  was  kept 
blind  to  the  earlier  efforts.  Only  the  particulars  about  the  Maverick  mission,  and  the  de¬ 
pendent  variables  used  to  measure  their  performance,  were  provided  during  model  de¬ 
velopment  and  testing.  The  results  of  the  other  efforts  were  made  available  only  after 
the  model  data  collection  was  completed. 

There  was  one  minor  error  in  the  information  transfer  for  preparing  the  model. 
We  were  told  that  pilots  could  not  release  their  Maverick  before  reaching  the  weapon’s 
maximum  range  of  57,000  yards.  The  Soar  model  was  programmed  to  incorporate  this 
constraint,  despite  the  fact  in  some  conditions  the  decision  to  fire  was  made  long  before 
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the  aircraft  reached  that  distance  from  the  target.  The  Soar  model  was  programmed  to 
delay  weapons  release  until  reaching  the  Maverick’s  maximum  range.  Unknown  to  our 
modeling  team,  pilots  in  the  HITL  simulation  often  released  their  missiles  before  reach¬ 
ing  the  missile  envelope  limits.  This  erroneous  behavior  is  understandable  because  the 
decision  to  fire  was  made  so  much  earlier  in  the  approach,  especially  in  the  enhanced 
cockpit  condition.  This  oversight  produced  an  artificially  induced  discrepancy  in 
weapon  release  range  between  the  Soar  model  and  the  HITL  simulation  results. 
Clearly,  equivalent  behaviors  were  thwarted  by  not  enforcing  the  Maverick  missile  re¬ 
lease  envelope  during  the  HITL  simulation. 

The  simplicity  of  the  Maverick  mission  did  not  afford  opportunity  to  explore  com¬ 
parative  measures  between  models  or  error  analysis.  Multiple  model  runs  can  be  used 
to  determine  probabilities  of  specific  errors  that  could  be  compared  across  competing 
models.  The  types  and  conditions  under  which  errors  occur  are  a  substantial  basis  for 
judging  model  fidelity.  Error  analysis  could  be  further  expanded,  at  least  in  Soar,  to 
look  at  the  antecedent  conditions  to  committing  the  error.  Soar  can  record  the  opera¬ 
tions  of  its  elaboration/decision  cycle  so  that  runs  resulting  in  errors  can  be  dissected  to 
determine  why  the  errors  occurred. 

Comparison  of  Cognitive  Models  to  Human  Performance 

The  true  potential  utility  of  cognitive  modeling  comes  in  its  comparison  to  human 
performance.  Early  comparisons  will  largely  be  made  to  tune  the  fidelity  of  Soar  based 
models  and  other  modeling  methods.  Most  of  these  comparisons  will  be  like  the  ones 
described  above  between  different  cognitive  models.  Comparative  dependent  meas¬ 
ures  will  be  identified  and  the  differences  between  model  and  human  performance  will 
be  made  much  as  it  has  been  done  here  for  the  Maverick  mission.  Once  the  models 
have  been  tuned  to  mimic  their  human  counterparts,  cognitive  models  will  be  ready  to 
serve  as  the  first  significant  tools  for  cognitive  engineering. 

Engineering  involves  the  systematic  application  of  codified  knowledge  to  permit 
design  of  systems  with  confidence  in  the  ultimate  performance.  What  has  previously 
passed  as  cognitive  engineering  has  been  nothing  more  than  artful  application  of  cog- 
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nitive  science  to  the  system  design  process.  Cognitive  information  has  not  been  codi¬ 
fied  sufficiently  to  allow  its  application  in  a  systematic  fashion.  Thus,  cognitively  derived 
designs  lack  our  confidence  and  sometimes  do  not  out  perform  traditional  designs. 
Cognitive  models  may  codify  our  knowledge  in  such  a  way  to  allow  reliable  prediction 
about  a  proposed  design’s  ultimate  performance. 

Perhaps  the  most  important  use  of  cognitive  models  in  the  engineering  process 
will  be  the  discovery  of  design-based  errors  made  by  operators.  Cognitive  models  can 
be  used  to  conduct  large  numbers  of  mission  simulations  under  wider  ranges  of  condi¬ 
tions  than  are  feasible  with  HITL  simulations.  Environmental  stressors  (like  fatigue, 
temperature,  terrain,  weather),  unusual  events,  and  all  combinations  of  system  failures 
could  be  simulated  sufficient  numbers  of  times  to  identify  and  establish  probabilities  of 
errors  as  well  as  capture  the  rare  combinations  of  events  leading  up  to  catastrophic 
failures.  Furthermore,  Soar  models  can  provide  the  diagnostic  information  about  how 
the  cognitive  events  precipitate  error.  These  failures  could  be  verified  by  programming 
their  antecedent  events  into  HITL  simulations  to  validate  the  cognitive  modeling  results. 
These  kinds  of  error  analyses  could  be  powerful  predictive  tools  for  cognitive  engineers, 
enabling  virtual  test  of  system  interface  designs  before  they  are  actually  Implemented. 
Cognitive  models  hold  the  potential  to  put  the  engineering  into  cognitive  engineering. 

RECOMMENDATIONS 

The  present  modeling  effort  establishes  that  Soar  can  model  the  same  kinds  of 
tasks  that  discrete  event  models  can,  and  with  an  enhanced  perceptual  representation 
perform  as  well  or  better  than  discrete  event  simulations  like  EADSIM.  The  particular 
mission  given  to  be  modeled  did  not  exercise  Soar’s  cognitive  modeling  capabilities  be¬ 
yond  the  perceptual  capabilities.  The  principle  recommendation  is  that  Soar  should  be 
applied  to  more  cognitively  demanding  tasks,  either  tasks  that  span  multiple  features  of 
cognitive  processing  or  suites  of  tasks  that  focus  on  particular  aspects  of  cognitive 
processing. 

Perhaps  the  best  way  to  approach  creating  a  comprehensive  cognitive  model  is 
to  model  “slices”  of  cognitive  behavior  (the  part-task  approach)  which  are  constrained 
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by  limited  input,  outputs,  and  event  complexity.  Such  limited  models  could  serve  as  test 
beds  where  the  effects  of  the  anthropomorphizing  modifications  could  be  tested  and 
compared  with  similarly  constrained,  HITL  simulations.  These  limited  scope  models 
should  be  designed  so  that  they  could  be  linked  either  in  serial  or  later  in  parallel  so  that 
interactions  in  their  operations  can  be  compared  to  their  earlier  function  in  isolation.  In 
this  fashion,  a  more  comprehensive  model  of  operator  function  could  be  built  while  ex¬ 
ploring  the  basic  science  issues  of  modeling  human  cognition. 

Model  elaboration  would  eventually  produce  a  comprehensive  model  of  a  human 
operator  that  could  be  tested  in  a  fashion  suitable  for  cognitive  engineering  predictions 
of  performance.  More  complex  event  scripts  and  more  cognitive  demanding  events 
could  be  introduced  so  that  the  error  analysis  discussed  earlier  could  be  performed.  In 
this  way,  definitive  predictions  could  be  made  about  where  human  errors  can  be  antici¬ 
pated  and  the  antecedent  events  could  be  used  to  prepare  HITL  simulations  to  discover 
the  hypothesized  error  points.  Data  could  also  be  recovered  from  devices  like  flight  re¬ 
corders  or  from  accident  investigations  to  determine  whether  the  predicted  errors  are 
encountered  in  real  systems.  The  ultimate  goal  of  such  efforts  would  be  to  create  an 
engineering  procedure  by  which  operator  stations  can  be  thoroughly  modeled  and 
tested  so  that  cognitive  engineering  becomes  less  of  an  art  and  more  of  an  engineering 
science. 
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0 

0 

0 

14 

92744.6 

92472.5 

51 

2 

0 

0 

4 

6 

94010.2 

98285.6 

0 

0 

0 

9 

91485.6 

96429.9 

52 

2 

0 

0 

0 

3 

97272.5 

101948 

0 

0 

0 

4 

96121.6 

101066 

53 

2 

0 

0 

1 

3 

97203.6 

101486 

0 

0 

0 

5 

94849.5 

99790.5 

54 

2 

0 

0 

1 

8 

94620  ’ 

97420 

0 

0 

0 

9 

93557.7 

96628.9 

55 

2 

0 

0 

2 

6 

95816.7 

99026.6 

0 

0 

0 

9 

93626.6 

96698.7 

56 

2 

0 

0 

4 

17 

90456.1 

89898.7 

0 

0 

0 

20 

88334.7 

87538 

57 

2 

0 

0 

4 

11 

90462.6 

94928.2 

0 

0 

0 

14 

88485.6 

93036.4 

58 

2 

0 

0 

0 

13 

92692.1 

92311.8 

0 

0 

0 

15 

91465.9 

91367.5 

59 

2 

0 

0 

1 

3 

96367.6 

101043 

0 

0 

0 

5 

95259.4 

99806.9 

60 

2 

0 

0 

1 

10 

95731.5 

95783.9 

0 

0 

0 

15 

92042.9 

91947.9 

61 

2 

0 

0 

0 

14 

92633.1 

92085.6 

0 

0 

0 

15 

91590.5 

91318.4 

62 

2 

0 

0 

1 

13 

93059.3 

92679 

0 

0 

0 

15 

91446.2 

91344.6 

99 

2 

0 

0 

3 

9 

93597.1 

96400.3 

0 

0 

0 

11 

91639.7 

94715.1 

100 

2 

0 

0 

0 

3 

97082.3 

101758 

0 

0 

0 

4 

95947.9 

100889 

101 

2 

0 

0 

1 

IS 

92357.7 

91285.6 

0 

0 

0 

17 

91000.3 

90206.9 

102 

2 

0 

0 

0 

7 

95721.6 

98524.9 

0 

0 

0 

8 

94839.7 

97911.8 

103 

2 

0 

0 

2 

10 

95757.7 

95216.7 

0 

0 

0 

12 

94685.6 

93892.1 

104 

2 

0 

0 

3 

14 

92974.1 

92423.3 

0 

0 

0 

17 

90711.8 

89918,4 

105 

2 

0 

0 

1 

10 

91983.9 

96252.8 

0 

0 

0 

11 

90688.8 

95239.7 

106 

2 

0 

0 

0 

15 

91492.1 

91108.5 

0 

0 

0 

16 

90747.9 

90649.5 

107 

2 

0 

0 

1 

13 

94213.4 

93669.2 

0 

0 

0 

14 

93092.1 

92823.3 

108 

2 

0 

0 

0 

14 

93167.5 

92793.8 

0 

0 

0 

15 

91846.2 

91751.1 

109 

2 

0 

0 

0 

5 

100499 

99970.8 

0 

0 

0 

6 

99479 

99210.2 

110 

2 

0 

0 

2 

16 

91302 

90223.3 

0 

0 

0 

19 

86426.5 

88154.4 

147 

2 

0 

0 

2 

9 

93856.1 

97059.4 

0 

0 

0 

11 

92397 

95465.9 

148 

2 

0 

0 

0 

6 

94416.7 

99082.3 

0 

0 

0 

7 

93403.6 

98344.6 

149 

2 

0 

0 

3 

12 

90108.5 

94574.1 

0 

0 

0 

15 

86997 

91938 

150 

2 

0 

0 

1 

12 

94344.6 

94397.1 

0 

0 

0 

13 

93170.8 

93498.7 

151 

2 

0 

0 

1 

16 

67682.3 

90452.8 

0 

0 

0 

18 

85833.1 

88902 

152 

2 

0 

0 

4 

15' 

91262.6 

90708.5 

0 

0 

0 

18 

89679 

88885.6 

177 

2 

0 

0 

2 

17 

87557.7 

90334.7 

0 

0 

0 

19 

85446.2 

88511.8 

178 

2 

0 

0 

1 

6 

99295.4 

99357.7 

0 

0 

0 

8 

97941.3 

97843 

179 

2 

0 

0 

3 

9 

91531.5 

96000.3 

0 

0 

0 

12 

89728.2 

94275.7 

180 

2 

0 

0 

4 

11 

94702 

94751.2 

0 

0 

0 

15 

91770.8 

91675.7 

181 

2 

0 

0 

2 

13 

93521.6 

92974.1 

0 

0 

0 

14 

92869.2 

92597 

182 

2 

0 

0 

3 

19 

89370.8 

88285.6 

0 

0 

0 

22 

86633.1 

86361 

220 

2 

0 

0 

0 

3 

96587.2 

101259 

0 

0 

0 

4 

95429.8 

100371 

221 

2 

0 

0 

2 

18 

90774.1 

89505.2 

0 

0 

0 

20 

87702 

88023.3 

226 

2 

0 

0 

3 

5 

100584 

99800.4 

0 

0 

0 

8 

98374.1 

97577.4 

227 

2 

0 

0 

1 

10 

95672.5 

95728.2 

0 

0 

0 

11 

95059.4 

95387.2 

228 

2 

0 

0 

0 

18 

90383.9 

89305.2 

0 

0 

0 

19 

89302 

88508.5 

229 

2 

0 

0 

3 

12 

94741.3 

93482.3 

0 

0 

0 

14 

91902 

92223.3 

230 

2 

0 

0 

3 

12 

90862.6 

94052.8 

0 

0 

0 

15 

88951.1 

92016.7 

231 

2 

0 

0 

3 

7 

93423.3 

98092.1 

0 

0 

0 

10 

91459.3 

96010.2 

232 

2 

0 

0 

3 

11 

95761 

94964.3 

0 

0 

0 

14 

92613.4 

92344.6 

233 

2 

0 

0 

2 

7 

98554.4 

97305.3 

0 

0 

0 

12 

93872.5 

93770.8 

234 

2 

0 

0 

4 

12 

90879 

94072.5 

0 

0 

0 

15 

89220 

92282.3 

235 

2 

0 

0 

1 

9 

97167.6 

96629.9 

0 

0 

0 

10 

96252.8 

95983.9 
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RTIC-3 

FINAL  DATA 

second  designation 

first  lock 

second  lock 

launch  | 

range  to 

range  to 

range  to 

range  to 

time 

range 

airfield 

time 

range 

airfield 

time 

range 

airfield 

time 

range 

airfield 

18.25 

87352 

89149 

21.25 

85118 

86712 

22.25 

84408 

85897 

54.78 

57894 

59332 

4.47 

4285 

3667 

4.47 

3377 

3640 

4.47 

4263 

3654 

2.71 

36 

2247 

10 

92439,7 

95511.8 

13 

88436.4 

93085.6 

14 

89511.8 

92295.4 

51 

57862.5 

62357.6 

27 

76806.8 

81754,4 

30 

79469.1 

79472.4 

31 

74023.2 

78620 

56 

57915 

57777.3 

19 

89547.9 

88754.4 

22 

81442.9 

86069.2 

23 

86642.9 

85551.1 

51 

57915 

62410.1 

14 

92374.1 

92698.7 

17 

87679 

90456.1 

18 

89413.4 

89446,2 

54 

57905,1 

60492 

16 

91895.4 

91102 

19 

85842.9 

88613.4 

20 

88934.7 

87849.5 

53 

57842.8 

60436.3 

22 

86872.4 

86079 

25 

83426.5 

83429.8 

26 

84020 

82918.3 

57 

57829.7 

57675.6 

23 

82705,2 

85767.5 

26 

79039.6 

83262.6 

27 

79642.9 

82380,6 

52 

57957,6 

62062.5 

13 

93849.5 

93754.4 

16 

86764.2 

91016.7 

17 

90488.8 

90102 

52 

57846.1 

61947.8 

17 

90026.5 

89757.7 

20 

82898.7 

87134.7 

21 

86931.5 

86361 

51 

57924.8 

62026.4 

23 

85515.1 

85416.7 

26 

80124.9 

82862.6 

27 

82767.5 

82351.1 

54 

57875.6 

60469.1 

16 

00938 

90669.2 

19 

85492,1 

88259.3 

20 

87915.1 

87351.1 

53 

57908.4 

60508.4 

19 

88544.6 

88442.9 

22 

86672.4 

86101.9 

23 

85498.7 

65092.1 

57 

57898.6 

57174 

15 

88459.3 

91528.2 

18 

84383.9 

89020 

19 

85495.4 

88259.3 

51 

57875.6 

62370.7 

11 

95121.6 

95446.2 

14 

88233.1 

62882.3 

15 

92259.3 

92302 

51 

57957.6 

62456 

15 

92502 

91708.5 

18 

84790.5 

89429.8 

19 

89685.6 

88600.3 

51 

57905.1 

62400.2 

14 

92380.7 

92702 

17 

87511.8 

90285.6 

18 

89479 

89511.8 

53 

57918.2 

60508,4 

16 

91849.5 

91056.1 

19 

85911.8 

88679 

20 

88888.8 

87803.6 

53 

57908,4 

60498.6 

26 

82531,4 

82849.5 

29 

81698,6 

80587.2 

30 

79787.2 

79774.1 

58 

57842.8 

56600.2 

20 

84961 

88023.3 

23 

81544.5 

85777.4 

24 

82226.5 

84977.4 

52 

57915 

62019.9 

19 

83410.1 

87961 

22 

86095.4 

85692,1 

23 

80574.1 

84800.3 

57 

57882.2 

57305.1 

9 

96649.5 

96380.7 

12 

89511.8 

93774.1 

13 

93829.8 

93288.9 

51 

57875.6 

61977.3 

19 

85639.6 

88705.2 

22 

86357,7 

85957.7 

23 

82547.8 

85298.7 

57 

57908.4 

57334.6 

19 

85111.8 

88174.1 

22 

86328.2 

85757.7 

23 

82095.4 

84842.9 

57 

57937.9 

57216.6 

20 

87633.1 

87361 

23 

85410.1 

85000.3 

24 

84626.5 

84049.5 

57 

57924.6 

57347.7 

17 

85079 

90016.7 

20 

84734.7 

87505.2 

21 

82085.5 

86708.5 

53 

57865.8 

60462,5 

8 

97403.6 

97728.2 

11 

90577.4 

95236.4 

12 

94403.6 

94452.8 

51 

57941.2 

62436.3 

22 

81282.3 

86223.3 

25 

84944.6 

83846.2 

26 

78344.5 

82954.4 

58 

57957.6 

56715 

13 

93462.6 

93787.2 

16 

88688.8 

91465.9 

17 

90452.6 

90488.8 

54 

57878.9 

60465.8 

18 

85751.1 

88816.7 

21 

87564.2 

86475.7 

22 

82911.8 

85665.9 

58 

57888.7 

56646.1 

22 

85551.1 

85872.4 

25 

84659.3 

83561 

26 

82672.4 

82675.7 

58 

57862.5 

56619.9 

16 

88259.3 

91324.9 

19 

84744.6 

88990.5 

20 

85249.5 

88013.4 

52 

57859.2 

61960,9 

20 

82905.2 

87456 

23 

85357.7 

84947.8 

24 

79892.1 

84115.1 

57 

57911.7 

57334.6 

19 

84190.5 

88738 

22 

86918.3 

86351.1 

23 

81095.4 

85328.2 

58 

57908.4 

57187.1 

19 

85410.1 

88472.4 

22 

86656 

86256 

23 

82685.5 

85436.4 

57 

57839.5 

57265.8 

10 

92869.2 

95934.8 

13 

94079 

93531.5 

14 

89629.8 

92410.2 

57 

57898.6 

57177.2 

23 

85069.2 

84964.2 

26 

83075.7 

82492.1 

27 

81911.8 

81488.8 

57 

57898.6 

57177.2 

16 

86492.1 

91433.1 

19 

86183.9 

88951.1 

20 

83347.8 

87980.6 

54 

57849,4 

60436.3 

18 

89249.5 

89564.2 

21 

82269.1 

86895.4 

22 

86262.6 

86275.7 

51 

57852.7 

62344.5 

20 

88842.9 

88049.5 

23 

80813.4 

85436.4 

24 

85574.1 

84479 

51 

57869.1 

62364.2 

18 

86492.1 

89551.1 

21 

86915.1 

86941.3 

22 

83397 

86147.8 

57 

57872.3 

57728.1 

22 

86547.8 

85754.4 

25 

80675.7 

83416.7 

26 

83616.7 

82511.8 

53 

57924.8 

60515 

24 

83790.5 

84105.2 

27 

82852.8 

81744.6 

26 

80862.6 

80856 

58 

57951 

56708.4 

24 

80121.6 

84672.4 

27 

79597 

82334.7 

28 

77269.1 

81485.5 

54 

57898,6 

60495.3 

12 

90062.6 

94610.2 

15 

92498.7 

92121.6 

16 

87088.8 

91341.3 

58 

57833 

57259.2 

20 

87967.5 

87695.4 

23 

81026.5 

85256 

24 

85085.5 

84508.5 

51 

57944.5 

62046.1 

21 

83711.8 

86774.1 

24 

84941.3 

84531.4 

25 

80869.1 

83610,1 

57 

57937.9 

57364.1 

20 

84593.7 

87656 

23 

85708.5 

85134.7 

24 

81620 

84364.2 

57 

57842.8 

57121.5 

27 

82544.6 

82442.9 

30 

80403.6 

79806.8 

31 

79456 

79023,2 

58 

57872.3 

57151 

11 

91754.4 

94823.3 

14 

87665.9 

92315.1 

15 

88744.6 

91524.9 

51 

57852.7 

62347.8 

25 

79052.7 

83993.7 

28 

81498.6 

81495.4 

29 

76019,9 

80623.2 

57 

57846.1 

57698.6 

13 

88747.9 

93688.9 

16 

92216.7 

91141.3 

17 

85875.7 

90518,4 

58 

57855.9 

56613.3 

16 

88029.8 

91095.4 

19 

88865.9 

88895.4 

20 

85295.4 

88056 

57 

57921.5 

57774 

23 

81970.8 

85033.1 

26 

84242.9 

83141.3 

27 

79210.1 

81944.6 

58 

57941.2 

56698.6 

20 

88180.6 

87387.2 

23 

84918.3 

84928.2 

24 

85380.6 

84282.3 

56 

57928.1 

57777.3 

20 

83472.4 

88023.3 

23 

82541.3 

85288.8 

24 

80400.3 

84626.5 

54 

57888.7 

60482.2 

22 

86301.9 

86203.6 

25 

79652.7 

83879 

26 

83501.9 

83088.8 

52 

57895.3 

61996.9 

19 

83896.7 

88449.5 

22 

86501.9 

85931.5 

23 

80918.3 

85151.1 

57 

57944.5 

57219.9 

16 

87347.9 

90416.7 

19 

88557.7 

88161 

20 

84669.1 

87433.1 

57 

57915 

57337.9 

21 

87387.2 

87115.1 

24 

81993.7 

84741.3 

25 

84646.2 

84069.1 

54 

57849.4 

60442.8 

18 

89580.6 

89482.3 

21 

87501.9 

86934.7 

22 

86724.9 

86324.9 

58 

57882.2 

57160.9 
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TIC-5  FINAL  DATA 


trial 

number 

accuracy 

first  detection 

first  ID 

first  designation 

hits 

miss 

fa 

cr 

time 

range 

range  to 
airfield 

time 

range 

range 

to 

airfield 

time 

range 

range  to 
airfield 

average  2.00 

0.00 

0.00 

3.80 

10.10 

93806 

95613 

0 

0 

0 

13.90 

90554 

92727 

stdv  0.00 

0.00 

0.00 

1.86 

4.61 

3459 

3793 

0 

0 

0 

5.14 

3610 

4179 

15 

2 

0 

0 

5 

12 

91331.5 

93636.4 

0 

0 

0 

18 

86659.3 

89452.8 

16 

2 

0 

0 

5 

9 

92377.4 

96174.1 

0 

0 

0 

12 

89262.6 

94206.9 

17 

2 

0 

0 

5 

10 

96285.6 

95488.9 

0 

0 

0 

13 

93967.5 

93174.1 

249 

2 

0 

0 

2 

6 

97029.9 

99561 

0 

0 

0 

7 

95918.4 

98711.8 

19 

2 

0 

0 

4 

9 

94049.5 

96846.2 

0 

0 

0 

14 

90082.3 

92872.5 

20 

2 

0 

0 

5 

13 

93944.6 

93141.3 

0 

0 

0 

17 

90610.2 

89816.7 

21 

2 

0 

0 

5 

11 

91842.9 

95039.7 

0 

0 

0 

14 

89724.9 

92518.4 

22 

2 

0 

0 

1 

13 

893577 

93315.1 

0 

0 

0 

14 

88252.8 

9249B.7 

23 

2 

0 

0 

3 

7 

94111.8 

97911.8. 

0 

0 

0 

10 

91344.6 

95590.5 

24 

2 

0 

0 

2 

17 

87423.3 

90190.5 

0 

0 

0 

19 

86249.5 

89036.4 

25 

2 

0 

0 

6 

12 

91541.3 

94331.5 

0 

0 

0 

17 

87334.7 

90124.9 

26 

2 

0 

0 

4 

20 

88488.8 

87924.9 

0 

0 

0 

24 

84338 

84583.9 

63 

2 

0 

0 

3 

11 

91790.5 

94583.9 

0 

0 

0 

14 

89761 

92554.4 

64 

2 

0 

0 

2 

10 

91511.8 

96174.1 

0 

0 

0 

11 

90446.2 

95390.5 

65 

2 

0 

0 

4 

10 

91702 

95495.4 

0 

0 

0 

15 

87098.7 

92042.9 

66 

2 

0 

0 

4 

13 

93518.4 

93144.6 

0 

0 

0 

18 

89324.9 

89649.5 

67 

2 

0 

0 

5 

9 

94229.8 

97026.6 

0 

0 

0 

14 

90374.1 

93167.5 

68 

2 

0 

0 

3 

15 

92115.1 

92157.7 

0 

0 

0 

16 

90993.8 

91318.4 

69 

2 

0 

0 

6 

11 

90128.2 

93918.4 

0 

0 

0 

15 

87187.2 

91433.1 

70 

2 

0 

0 

4 

3 

97462.6 

101276 

0 

0 

0 

6 

94390.5 

98639.7 

71 

2 

0 

0 

4 

3 

97265.9 

101548 

0 

0 

0 

10 

91875.7 

96121.6 

72 

2 

0 

0 

3 

,  3 

98528.2 

101748 

0 

0 

0 

6 

96502 

99298.7 

73 

2 

0 

0 

4 

11 

95324.9 

94783.9 

0 

0 

0 

19 

88505.2 

88754.4 

74 

2 

0 

0 

7 

19 

89308.5 

88747.9 

0 

0  ■ 

0 

24 

84475.7 

84724.9 

111 

2 

0 

0 

3 

12 

91098.7 

94295.4 

0 

0 

0 

16 

88498.7 

91288.8 

112 

2 

0 

0 

2 

3 

96990.5 

101663 

0 

0 

0 

4 

95938 

100879 

113 

2 

0 

0 

2 

6 

95265.9 

99239.7 

0 

0 

0 

8 

92711.8 

97652.8 

114 

2 

0 

0 

4 

15 

92443 

92121.6 

0 

0 

0 

18 

89269.2 

89593.8 

115 

2 

0 

0 

4 

8 

97918.4 

97652.8 

0 

0 

0 

16 

91629.8 

90836.4 

116 

2 

0 

0 

4 

6 

99013.5 

98652.8 

0 

0 

0 

8 

96849.5 

97170.8 

117 

2 

0 

0 

3 

8 

91662.6 

96324.9 

0 

0 

0 

11 

90328.2 

94577.4 

118 

2 

0 

0 

7 

4 

97318.4 

101128 

0 

0 

0 

11 

91052.8 

95298.7 

119 

2 

0 

0 

3 

11 

95931.5 

94865.9 

0 

0 

0 

14 

92629.8 

92879 

120 

2 

0 

0 

4 

7 

98436.4 

98134.8 

0 

0 

0 

9 

96944.6 

96852.8 

121 

2 

0 

0 

2 

17 

89941.3 

89902 

0 

0 

0 

18 

89144.6 

89393.8 

122 

2 

0 

0 

4 

7 

98747.9 

98003.6 

0 

0 

0 

21 

87187.2 

87088.8 

183 

2 

0 

0 

0 

7 

94079 

98744.6 

0 

0 

0 

8 

92964.3 

97905.3 

184 

2 

0 

0 

6 

6 

99233.1 

98938.1 

0 

0 

0 

9 

96197.1 

96521.7 

185 

2 

0 

0 

1 

7 

94121.6 

97924.9 

0 

0 

0 

10 

90642.9 

95583.9 

186 

2 

0 

0 

1 

11 

92557.7 

95072.5 

0 

0 

0 

12 

91534.8 

94324.9 

187 

2 

0 

0 

7 

7 

94941.3 

97747.9 

0 

0 

0 

13 

90134.7 

92928.2 

188 

2 

0 

0 

6 

11 

95305.3 

94997.1 

0 

0 

0 

16 

90964.3 

91288.8 

189 

2 

0 

0 

6 

7 

93331.5 

98000.3 

0 

0 

0 

16 

87144.6 

91393.8 

190 

2 

0 

0 

2 

3 

97849.5 

101830 

0 

0 

0 

4 

96718.4 

100964 

191 

2 

0 

0 

5 

9 

97216.7 

96157.7  ■ 

0 

0 

0 

17 

89459.3 

89711.8 

192 

2 

0 

0 

4 

4 

98180.7 

100994 

0 

0 

0 

8 

94882.3 

97675.8 

193 

2 

0 

0 

2 

16 

91692.1 

90882.3 

0 

0 

0 

19 

88377.4 

88626.5 

194 

2 

0 

0 

6 

10 

94734.8 

94783.9 

0 

0 

0 

16 

90718.4 

90616.7 

236 

2 

0 

0 

8 

16 

86115.1 

90564.3 

0 

0 

0 

21 

81564.2 

86505.2 

237 

2 

0 

0 

4 

5 

94774.1 

99446.2 

0 

0 

0 

6 

94016.7 

98961 

238 

2 

0 

0 

4 

16 

90308.5 

90272.5 

0 

0 

0 

22 

86675.7 

85882.3 

239 

2 

0 

0 

5 

12 

94046.2 

93675.7 

0 

0 

0 

16 

90262.6 

905872 

240 

2 

0 

0 

1 

8 

98210.2 

97154.4 

0 

0 

0 

9 

974462 

96652.8 

241 

2 

0 

0 

4 

21 

87036.4 

86636.4 

0 

0 

0 

23 

85167.5 

85485.5 

242 

2 

0 

0 

2 

20 

84367.5 

87131.5 

0 

0 

0 

22 

82879 

85672.4 

243 

2 

0 

0 

1 

8 

93167.5 

97138 

0 

0 

0 

10 

92000.3 

96249.5 

244 

2 

0 

0 

0 

5 

95708.5 

99685.6 

0 

0 

0 

6 

944692 

98715.1 

246 

2 

0 

0 

7 

12 

94898.7 

94141.3 

0 

0 

0 

16 

91472.5 

91377.4 

247 

2 

0 

0 

5 

9 

97715.1 

96656.1 

0 

0 

0 

15 

91823.3 

92072.5 

248 

2 

0 

0 

3 

15 

93318.4 

92059.3 

0 

0 

0 

19 

88849.5 

88751.1 

45 


RTIC-5 


second  designation  first  lock 


FINAL  DATA 


second  lock  launch 


range  to 
time  range  airfield 


range  to 
airfield 


20.55 

85911 

87350 

23.55 

83059 

84918 

24.55 

82957 

84082 

54.43 

57909 

59623 

4.51 

4441 

3666 

4.51 

3388 

3689 

4.51 

4427 

3674 

2.64 

37 

2064 

23 

80147.8 

85092.1 

26 

80521.6 

82990.5 

27 

77501.9 

82108.5 

54 

57908.4 

60236.3 

21 

86652.8 

86977.4 

24 

79964.2 

84583.9 

25 

83833.1 

83846.2 

51 

57921.5 

62419.9 

20 

82479 

87420 

23 

86187.2 

85095.4 

24 

79377.3 

83993.7 

58 

57892 

56649.4 

13 

93380.7 

93708.5 

16 

89006.9 

91508.5 

17 

90305.2 

90344.6 

54 

57921.5 

60239.6 

20 

88813.4 

88020 

23 

83141.3 

85616.7 

24 

85738 

84639.6 

54 

57918.2 

60239.6 

23 

84656 

84974.1 

26 

83134.7 

82029.8 

27 

81305.2 

81301.9 

58 

57833 

56590.4 

21 

82757.7 

87003.6 

24 

82147.8 

84626.5 

25 

79685.5 

83606.9 

54 

57937.9 

60262.5 

19 

88570.8 

88472.4 

22 

82010.1 

85941.3 

23 

85577.4 

85174.1 

52 

57849.4 

61646.1 

15 

91285.6 

91534.8 

18 

84970.8 

88911.8 

19 

88233.1 

88187.2 

51 

57905.1 

61701.9 

24 

84990.5 

84895.4 

27 

80147.8 

82610.1 

28 

81892.1 

81475.7 

54 

57859.2 

60177.3 

23 

85079 

85328.2 

26 

80452.7 

82918.3 

27 

82088.8 

82023.2 

54 

57947.7 

60272.3 

29 

80734.7 

80633.1 

32 

77872.4 

77787.2 

33 

77865.9 

77429.6 

57 

57855.9 

57646.1 

19 

83629.8 

88570.8 

22 

83701.9 

86187.2 

23 

80528.2 

85147.8 

54 

57947.7 

60272.3 

19 

88580.6 

88908.5  ■ 

22 

81659.3 

86288.8 

23 

85646.2 

85669.2 

51 

57954.3 

62449.4 

19 

89485.6 

88692.1 

22 

81577.3 

86203.6 

23 

86449.5 

85354.4 

51 

57878.9 

62374 

22 

83646.2 

86433.1 

25 

83977.3 

83990.5 

26 

80410.1 

82872.4 

57 

57878.9 

57731.4 

20 

89029.8 

88236.4 

23 

83020 

85495.4 

24 

86111.8 

85016.7 

54 

57908.4 

60229.7 

23 

86361 

85567.5 

26 

83298.7 

83308.5 

27 

83642.9 

82541.3 

57 

57905.1 

57764.1 

22 

82993.7 

85783.9 

25 

79383.9 

83305.2 

26 

79960.9 

82426.5 

51 

57869.1 

61669.1 

12 

93934.8 

93839.7 

15 

87403.6 

91354.4 

16 

91026.6 

90642.9 

51 

57954.3 

61754.3 

14 

92465.9 

92718.4 

17 

86393.8 

90341.3 

18 

89636.4 

89600.3 

52 

57865.8 

61662.5 

11 

95210.2 

95115.1 

14 

90256.1 

92764.3 

15 

92302 

91924.9 

54 

57957.6 

60282.2 

24 

81957.7 

84741.3 

27 

82305.2 

82239.6 

28 

78862.6 

81318.3 

57 

57908.4 

57701.8 

31 

79029.8 

78928.1 

34 

76692.1 

76600.3 

35 

76305.2 

75862.6 

57 

57944.5 

57737.9 

20 

83177.3 

88121.6 

23 

83210.1 

85692.1 

24 

80269.1 

84888.8 

54 

57941.2 

60265.8 

15 

91410.2 

91731.5 

18 

84941.3 

89577.4 

19 

88721.6 

88751.1 

51 

57918.2 

62413.3 

18 

90180.6 

89387.2 

21 

82390.5 

87020 

22 

87462.6 

86374.1 

51 

57908.4 

62403.5 

25 

81138 

83924.9 

28 

81728.2 

81728.2 

29 

78098.6 

80554.4 

57 

57875.6 

57728.1 

21 

84069.1 

86859.3 

24 

85934.7 

84839.6 

25 

81505.2 

83977.3 

58 

57869.1 

56626.4 

15 

92288.9 

91495.4 

18 

88977.4 

89003.6 

19 

89229.8 

88144.6 

56 

57944.5 

57796.9 

16 

87816.7 

90606.9 

19 

84193.7 

88131.5 

20 

84892.1 

87377.4 

51 

57934.6 

61731.4 

20 

88108.5 

88013.4 

23 

81724.9 

85652.8 

24 

85200.3 

84797 

52 

57895.3 

61695.3 

19 

84593.7 

88842.9 

22 

86521.6 

86472.4 

23 

81685.5 

85613.4 

57 

57846.1 

57636.3 

19 

86157.7 

88944.6 

22 

86626.5 

86229.8 

23 

83000.3 

85475.7 

58 

57951 

57383.8 

24 

81662.6 

84449.5 

27 

82177.3 

82111.8 

28 

78905.2 

81364.2 

57 

57918.2 

57708.4 

26 

82642.9 

82888.8 

29 

80964.2 

80541.3 

30 

79642.9 

79564.2 

57 

57836.3 

57265.8 

13 

90997 

93790.5 

16 

86711.8 

91357,7 

17 

88121.6 

90620 

51 

57951 

62446.1 

17 

84990.5 

89931.5 

20 

87590.5 

87613.4 

21 

82006.8 

86633.1 

56 

57951 

57803.5 

14 

93400.3 

92606.9 

17 

85442.9 

90082.3 

18 

90170.8 

89092.1 

51 

57905.1 

62400.2 

17 

89987.2 

90311.8 

20 

85485.5 

87974.1 

21 

87220 

87246.2 

54 

57934.6 

60259.2 

21 

87252.8 

86459.3 

24 

81656 

84131.4 

25 

84092.1 

82990.5 

53 

57872.3 

60196.9 

22 

87354.4 

86561 

25 

83888.8 

83901.9 

26 

84393.7 

83292.1 

57 

57839,5 

57695.3 

22 

83554.4 

86338 

25 

80118.3 

84042.9 

26 

80810.1 

83275.7 

52 

57937.9 

61737.9 

14 

92875.7 

92780.7 

17 

86433.1 

90380.6 

18 

89879 

89495.4 

52 

57915 

61715 

24 

79810.1 

84056 

27 

81469.1 

81400.3 

28 

77118.3 

81029.8 

56 

57954.3 

57747.7 

21 

87442.9 

87344.6 

24 

82393.7 

84869.2 

25 

84505.2 

84098,7 

54 

57957.6 

60278.9 

23 

82659.3 

85446.2 

26 

83115 

83052.8 

27 

79859.3 

82321.6 

57 

57865.8 

57655.9 

22 

85541.3 

85790.5 

25 

83793.7 

83380.6 

26 

82334.7 

82272.4 

57 

57908.4 

57328.1 

29 

77246.2 

80036.4 

32 

73056 

77642.9 

33 

74315 

76754.4 

50 

57924.8 

62416.6 

22 

85767.5 

86088.8 

25 

78990.4 

83606.9 

26 

82842.9 

82849.5 

51 

57846.1 

62341.2 

26 

77990.4 

82931.4 

29 

81505.2 

80390.4 

30 

74938 

79534.7 

58 

57954.3 

56715 

22 

82901.9 

85692.1 

25 

83193.7 

83200.3 

26 

79990.4 

82456 

56 

57941.2 

57790.4 

14 

89961 

92754.4 

17 

91511.8 

90433.1 

18 

86905.2 

89403.6 

58 

57859.2 

56616.6 

30 

80675.7 

79882.2 

33 

77439.6 

77419.9 

34 

77583.9 

76456 

57 

57941.2 

57790.4 

27 

77419.9 

81665.9 

30 

76938 

79390.4 

31 

74626.5 

78524.9 

53 

57934.6 

60259.2 

17 

90465.9 

90370.8 

20 

84183.9 

88121.6 

21 

87606.9 

87210.1 

52 

57944.5 

61744.5 

22 

85983.9 

86233.1 

25 

79823.2 

83744.6 

26 

82774.1 

82711.8 

52 

57901.8 

61701.9 

24 

82151.1 

84941.3 

27 

82620 

82206.9 

28 

79242.9 

81705.2 

58 

57908.4 

57337.9 

22 

83649.5 

86433.1 

25 

83990.5 

83928.2 

26 

80711.8 

83177.3 

57 

57937.9 

57728.1 

23 

85436.4 

85685.5 

26 

83544.6 

83134.7 

27 

82469.1 

82403.6 

58 

57882.2 

57311.7 

BASELINE-3 


FINAL  DATA 


_ accuracy 

trial 

number  hits  miss 

average  1.90  0.12  0. 
stdv  0.30  0.32  0. 

27  2  0 

28  2  0 

29  2  0 

30  2  0 

31  2  0 

32  2  0 

33  2  0 

34  2  0 

35  2  0 

36  2  0 

37  2  0 

38  2  0 

75  2  0 

76  2  0 

77  2  0 

78  2  0 

79  2  0  ( 

80  1  1 

81  2  0 

82  2  0 

83  2  0 

84  2  0 

85  2  0 

86  2  0 

123  2  0 

124  1  0 

125  2  0 

126  2  0 

127  2  0 

128  2  0 

129  2  0  ( 

130  2  0  t 

131  2  0  C 

132  2  0  C 

133  2  0  C 

134  1  0  1 

153  2  0  C 

154  2  0  C 

155  2  0  C 

156  2  0  C 

157  2  0  C 

158  2  0  C 

159  1  0  1 

160  2  0  0 

161  2  0  0 

162  2  1  0 

163  2  0  0 

164  2  0  0 

195  2  0  0 

196  2  0  0 

197  2  0  0 

198  2  1  0 

199  2  1  0 

200  2  0  0 

201  2  1  0 

202  1  11 

203  1  0  1 

204  2  0  0 

205  2  0  0 

206  2  1  0 


first  detection 


first  ID 


time 

range 

9.97 

94522 

4.82 

3263 

12 

89564.2 

14 

92010.2 

18 

90423.3 

10 

96911.8 

9 

93265.9 

14 

92816.7 

3 

98075.8 

8 

92747.9 

8 

92820 

3 

98456.1 

15 

88944.6 

15 

91324.9 

4 

95679 

3 

97220 

11 

95829.8 

14 

93856.1 

11 

96108.5 

13 

93334.8 

9 

93577.4 

17 

90279 

16 

91275.7 

4 

97518.4 

15 

92662.6 

8 

98377.4 

5 

97016.7 

3 

97272.5 

13 

94170.8 

10 

92613.4 

17 

90383.9 

13 

94397.1 

3 

97357.7 

8 

97482.3 

11 

95583.9 

15 

91970.8 

9 

96967.6 

8 

98006.9 

12 

88767.5 

11 

91075.7 

19 

89482.3 

3 

98111.8 

12 

91220 

7 

99180.7 

4 

95747.9 

15 

92164.3 

6 

94646.2 

15 

92151.1 

12 

94849.5 

19 

88554.4 

6 

94387.2 

6 

99449.5 

11 

95679 

10 

96282.3 

14 

93551.1 

18 

90292.1 

6 

93944.6 

8 

98213.5 

5 

95613.5 

3 

101387 

4 

98033.1 

3 

102256 

range  to  I 


3934 

94026.6 

92049.5 

89341.3 

95662.6 

96469.2 

91744.6 
100889 

97410.2 

97292.1 
101676 

92131.5 
91361 
100158 
101892 

94767.5 

92593.8 
95043 

93377.4 

96374.1 

90315.1 

90721.6 
100328 

91590.5 

97321.7 

99826.6 
101754 

93370.8 

95406.9 

89826.5 

93141.3 
101840 

97115.1 

94521.6 

92006.9  . 

95908.5 
96761 

93226.6 

95344.6 

88400.3 
101332 

94010.2 

98652.8 
100420 

91783.9 

99124.9 
91774.1 

93783.9 

88580.6 

98662.6 

99511.8 

94882.3 

95918.4 

92747.9 
89479 
98420 

96970.8 
100286 
101036 
100843 
101735 


first  designatic 

range  to 

range 

airfield 

time 

range 

67123 

68195 

46.47 

64955 

1708 

2901 

3.90 

1941 

64600.2 

68951,1 

46 

61482.2 

65859.2 

65770.7 

48 

64839.6 

64924.8 

63731.4 

51 

63613.3 

68229.8 

66856 

48 

64964.2 

66429.7 

69501.9 

45 

63937.9 

66180.6 

64993.7 

49 

64793.7 

68816.6 

71492.1 

42 

66701.9 

66157.6 

70708.4 

43 

64856 

66203.5 

70564.2 

45 

63354.3 

69947.8 

73042.9 

40 

68688.8 

64236.3 

67295.3 

48 

62233 

65456 

65364.2 

51 

62918.3 

67701.9 

72069.1 

41 

65997 

69656 

74226.5 

38 

68656 

67682.2 

66501.9 

47 

67003.5 

66692 

65308.4 

50 

63364.2 

67878,9 

66701.9 

47 

66708,4 

66505.2 

66423.2 

51 

63616.6 

66554.3 

69213.4 

45 

64813.3 

64908.4 

64816.6 

51 

62911.7 

65429.7 

64760.9 

49 

64262.5 

68570.7 

71242.9 

43 

66124.8 

66197 

65010.1 

51 

62859.2 

69042.9 

67872.4 

48 

65551.1 

68275.6 

70944.5 

42 

67075.6 

68472.4 

72842.9 

40 

67206.8 

66918.3 

66003.5 

50 

64252.7 

66078.9 

68734.7 

44 

65311.7 

65033 

64360.9 

51 

63826.5 

66941.2 

65557.6 

51 

62806.8 

69816.6 

74193.7 

39 

68019.9 

68577.3 

68082.2 

45 

67462.5 

67570.7 

66393.7 

49 

64701.9 

65803.5 

65711.7 

49 

64803.5 

68269.1 

67092 

48 

65085.5 

68813.4 

67442.9 

47 

65485.5 

64226.5 

68574 

46 

61334.6 

65364,2 

69518.3 

45 

62954.3 

64554.3 

63357.6 

51 

63518.3 

68869.1 

71957.6 

45 

64646.1 

65505.1 

68154.3 

45 

64737.9 

69367.5 

68721.6 

46 

67869.1 

67767.5 

72328.1 

41 

66203.5 

65859.2 

65347.8 

49 

64308.4 

67115 

71478.9 

42 

65695.3 

65934.7 

65426.5 

52 

62229.7 

67269.1 

66088.8 

49 

64292 

64078.9 

63977.3 

51 

63029.7 

66990.4 

71151.1 

42 

64931.4 

69410.1 

69344.5 

44 

68537.9 

67701.9 

66793.7 

51 

63574 

67937.9 

67442.9 

51 

62557.6 

66547.8 

65629.7 

53 

62059.2 

64885.5 

63957.6 

51 

63390.4 

66728.1 

71088.8 

42 

65059.2 

68892 

67524.8 

49 

65426.5 

67646.1 

72206.8 

40 

67013.4 

70515 

70036.3 

43 

69406.8 

68790.4 

71459.3 

42 

66977.3 

70980.6 

70344.5 

46 

67265.8 

range  to 
airfield 

66435 

3188 

66426.5 

65147.8 

62819.9 

65275.6 

66983.8 

63996.9 
69754.3 

69406.8 

67901.9 

71741.2 

65278.9 

62800.2 

70941.2 

73600.3 
66210.1 

63672.4 
65915 

62619.9 

67862.5 

62796.9 

63990.4 
69174 

62587.1 

65278.9 

70121.6 

71754.4 

63459.2 

68357.6 
63033 
63115 

72570.8 

67344.5 

64429.7 

64685.5 

64813.3 
65797 

66275.6 

67898.6 

62724.8 
67692 

67783.8 

67075.6 

70954.4 

64193.7 

70246.1 
62115 

64019.9 

62911.7 

69875.7 

68849.4 

62780.6 

62865.8 

61265.8 

62596.9 

69610.1 

64436.3 

71957.6 

69295.3 

70023.2 

66993.7 
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BASELINE-S  FINAL  DATA 


second  designation 

first  lock 

second  lock 

launch 

range  to 

range  to 

range  to 

range  to 

time 

range 

airfield 

time 

range 

airfield 

tirne 

range 

airfield 

time 

range 

airfield 

51.72 

59994 

62164 

54.72 

58725 

59759 

55.73 

57187 

58899 

57.93 

56147 

57159 

3.37 

3727 

2767 

3.37 

1888 

2771 

3.38 

3679 

2739 

3.14 

1509 

2510 

52 

59013.3 

62052.7 

55 

55016.6 

59485.5 

56 

56315 

58888.7 

58 

52734.6 

57183.8 

52 

57003.5 

61947.8 

55 

59655.9 

59518.2 

56 

54298.6 

58764.1 

58 

57288.7 

57131.3 

55 

54737.9 

59682.2 

58 

58551 

57315 

59 

52023.1 

56469.1 

61 

56128.1 

54872.3 

52 

58908.4 

61941.2 

55 

59728.1 

59596.9 

56 

56111.7 

58682.2 

58 

57265.8 

57115 

50 

64131.4 

63337.9 

53 

58226.4 

60819.9 

54 

61301.9 

60085.5 

56 

56049.4 

58623.2 

55 

58646.1 

58947.8 

58 

57806.8 

56560.9 

59 

55960.9 

55790.4 

61 

55606.8 

54344.5 

46 

61970.7 

66518.3 

49 

61708.4 

64337.9 

50 

59124.8 

63236.3 

53 

57852.7 

60446.1 

48 

65275.6 

65157.6 

51 

58770.7 

62878.9 

52 

62390.4 

61852.7 

54 

56308.4 

60400.2 

51 

63219.9 

62947.8 

54 

56495.3 

60587.1 

55 

60262.5 

59557.6 

57 

54200.2 

58272.3 

49 

64665.8 

64551 

52 

59426.4 

62036.3 

53 

61833 

61292 

55 

57069.1 

59655.9 

53 

61656 

61383.8 

56 

56255.9 

58836.3 

57 

58892 

58177.3 

59 

53983.8 

56541.2 

55 

59767.4 

59492 

58 

57616.6 

57039.5 

59 

57003.5 

56272.3 

61 

55334.6 

54734.6 

48 

62226.4 

65265.8 

51 

58288.7 

62787.1 

52 

59472.3 

62075.6 

54 

55937.9 

60416.6 

46 

66665.8 

66977.3 

49 

60036.3 

64547.8 

50 

64033 

63931.4 

52 

57846.1 

62341.2 

51 

57918.2 

62862.5 

54 

61659.2 

60446.1 

55 

54947.7 

59416.6 

58 

57908.4 

56665.8 

54 

57344.5 

60377.3 

57 

58095.3 

57951 

58 

54610 

57167.4 

60 

55901.8 

55734.6 

53 

58118.2 

61157.6 

56 

59911.7 

58682.2 

57 

55292 

57859.2 

59 

57646.1 

56400.2 

56 

59354.3 

58560.9 

59 

57449.4 

55996.9 

60 

56515 

55259.2 

62 

55255.9 

53780.5 

50 

59092 

63639.6 

53 

58813.3 

61416.6 

54 

56413.3 

60501.9 

56 

56315 

58892 

55 

54921.5 

59472.3 

58 

57747.7 

57177.2 

59 

52377.2 

56433 

61 

55396.9 

54800.2 

54 

55406.8 

69957.6 

57 

58210 

57492 

58 

52682.2 

56737.9 

60 

55859.2 

55121.5 

49 

64580.6 

64465.8 

52 

59410 

62016.6 

53 

61777.3 

61239.6 

55 

57180.5 

59767.4 

55 

56272.3 

59311.7 

58 

57708.4 

56987.1 

59 

53682.2 

56239.5 

61 

55367.4 

54623.1 

52 

62134.6 

62016.6 

55 

59990.4 

59285.5 

56 

59357.6 

58800.2 

58 

57918.2 

57196.9 

47 

61262.5 

66206.8 

50 

61174 

63790.4 

51 

58069.1 

62567.4 

54 

57846.1 

60433 

46 

66449.4 

66757.6 

49 

60341.2 

64462.5 

50 

63692 

63587.1 

52 

57819.9 

61921.5 

54 

55410 

60354.3 

57 

59101.8 

57869.1 

58 

52898.5 

57351 

60 

56964.1 

55714.9 

48 

64859.2 

65164.2 

51 

59974 

62583.8 

52 

62056 

61937.9 

54 

57678.9 

60265.8 

57 

55010 

58036.3 

60 

56908.4 

55655.9 

61 

52262.5 

54790.4 

63 

54534.6 

53265.8 

55 

60678.9 

59885.5 

58 

57642.8 

57492 

59 

57819.9 

56577.2 

61 

55167.4 

54993.6 

43 

66278.9 

69328.1 

46 

62990.4 

67128.1 

47 

63315 

65954.3 

52 

57957.6 

62059.2 

52 

57203.5 

61751 

55 

59793.7 

59229.7 

56 

54357.6 

58429.7 

58 

57492 

56908.4 

54 

55836.3 

60387.1 

57 

58564.1 

57846.1 

59 

52151 

56203.5 

61 

55574 

54833 

54 

57324.8 

60367.4 

57 

58833 

58265.8 

58 

54806.8 

57374 

60 

56387.1 

55796.9 

53 

57574 

60616.6 

56 

59069.1 

58357.6 

57 

55088.7 

57662.5 

59 

56705.1 

55974 

52 

62226.4 

61951 

55 

59449.4 

59315 

56 

59252.7 

58541.2 

58 

57196.9 

57042.8 

52 

58255.9 

61292 

55 

54514.9 

58980.5 

56 

55387.1 

57954.3 

58 

52229.7 

56675.6 

51 

62770.7 

63078.9 

54 

56213.3 

60695.3 

55 

59924.8 

59793.7 

57 

53790.4 

58252.7 

57 

52892 

57833 

60 

56705.1 

55452.7 

61 

50259.2 

54685.4 

63 

54324.8 

53052.6 

61 

62662.5 

62964.2 

54 

57744.5 

60334.6 

55 

59823.2 

59688.7 

57 

55478.9 

58046.1 

51 

63852.7 

63059.2 

54 

58085.5 

60675.6 

55 

60980.5 

59760.9 

57 

55616.6 

58183.8 

52 

61783.8 

62092 

55 

60977.3 

59757.6 

56 

59010 

58875.6 

58 

57915 

56672.3 

47 

62934.6 

65980.6 

50 

59216.6 

63528.1 

51 

59783.8 

62396.9 

53 

56855.9 

61151 

53 

55996.9 

60544.5 

56 

59259.2 

58698.6 

57 

53721.5 

57787.1 

59 

56849.4 

56269.1 

46 

67128.1 

66856 

49 

60446.1 

64567.4 

50 

64449.4 

63774 

52 

57957.6 

62059.2 

57 

54983.8 

58023.2 

60 

56334.6 

55747.7 

61 

52337.9 

54878.9 

63 

53872.3 

53259.2 

54 

56921.5 

59967.4 

57 

58285.5 

57567.4 

58 

53751 

56311.7 

60 

56029.7 

55295.3 

55 

59892 

59619.9 

58 

57541.2 

56964.1 

59 

56990.4 

56259.2 

61 

55396.9 

54796.9 

48 

62069.1 

65115 

51 

58016.6 

62508.4 

52 

59121.5 

61724.8 

54 

55711.7 

60187.1 

48 

60478.9 

65419.9 

51 

63256 

63147.8 

52 

57685.4 

62177.3 

57 

57885.4 

57737.9 

56 

53878.9 

58823.2 

59 

57623.2 

56377.2 

60 

50954.3 

55387.1 

62 

55255.9 

53990.4 

65 

56603.5 

59633 

58 

57259.2 

57108.4 

59 

53875.6 

56419.9 

61 

54918.2 

54744.5 

57 

54678.9 

57705.1 

60 

56711.7 

55459.2 

61 

52023.1 

54547.7 

63 

54426.4 

53154.3 

55 

59160.9 

59459.2 

58 

58236.3 

56993.6 

59 

56292 

56124.8 

61 

55941.2 

54682.2 

50 

60062.5 

63108.4 

53 

56692 

60783.8 

54 

57275.6 

59869.1 

56 

54223.1 

58292 

53 

56423.1 

60974 

56 

60128.1 

58698.6 

57 

53977.2 

58046.1 

59 

57915 

56469.1 

52 

62174 

61898.6 

55 

55370.7 

59846.1 

56 

59528.1 

58819.9 

58 

52878.9 

57331.4 

48 

62242.8 

65288.8 

51 

63187.1 

62659.2 

52 

58964.1 

61567.4 

57 

57878.9 

57308.4 

47 

66397 

66124.8 

50 

61010.1 

63626.5 

51 

63396.9 

62715 

53 

57921.5 

60515 

52 

62249.4 

62131.4 

55 

60242.8 

59537.9 

56 

59249.4 

58688.7 

58 

57859.2 

57137.9 

48 


BASELINE'S 

_ accuracy 

trial 

number  hits  miss 


average  1,82  0.08  0. 
stdv  0.39  0.28  0. 

39  2  0 

40  2  0 

41  1  1 

42  2  0 

43  2  1 

44  2  0 

45  2  0 

46  2  0 

47  1  0 

48  2  0 

49  2  0 

50  2  0 

87  2  0 

88  1  0 

89  2  0 

90  1  1 

91  2  0 

92  2  0 

93  2  0 

94  2  0 

95  2  0 

96  2  0 

97  2  0 


FINAL  DATA 


Ifirst  detection 


first  ID 


Ifirst  designation 


range 

range  to 
airfield 

time 

range 

range  to 
airfield 

time 

range 

range  tc 
airfield 

94094 

95736 

43 

66938 

68460 

46.63 

64185 

66161 

3267 

4082 

4 

1760 

2948 

4.07 

2279 

3244 

91751.1 

95544.6 

42 

65642.9 

69324.8 

47 

60741.2 

65685,5 

96597.1 

100882 

39 

68141.2 

72311.7 

41 

66151.1 

71092.1 

94567.5 

99046.2 

40 

67105.2 

71469.1 

45 

63737.9 

67987.1 

93059.3 

93102 

46 

66419.9 

66334.7 

47 

65783.8 

66092 

89751.1 

89711.8 

48 

64665.8 

64508.4 

55 

60295.3 

59501.9 

88951.1 

88633.1 

49 

64278.9 

63846.1 

54 

61233 

60439.6 

93043 

96239.7 

43 

66315 

69383.9 

47 

63374 

66147.8 

93472.5 

92206.9 

47 

66603.5 

65216.6 

51 

62403.5 

62282.2 

94321.6 

98128.2 

40 

67006.8 

70695.3 

46 

62098.6 

66344.5 

89033.1 

91538 

45 

64295.3 

66675.6 

46 

63780.6 

66560.9 

92695.4 

91892.1 

48 

66180.6 

65262.5 

51 

62728.1 

62967.4 

96452.8 

96429.9 

44 

68036.3 

67895.3 

45 

67167.5 

67406.8 

96459.4 

100941 

38 

68056 

72423.2 

41 

65475.6 

70416.6 

97495.4 

101309 

37 

69564.2 

73265.8 

39 

67970.7 

72219.9 

90872.5 

90583.9 

48 

65200.2 

64793.7 

51 

63600.2 

62806.8 

96190.5 

95439.7 

46 

67875.6 

67003.5 

47 

66793.7 

66305.2 

95010.2 

94213.4 

46 

67324.8 

66413,4 

48 

65521.5 

64728.1 

95170.8 

94902 

45 

67370.7 

66980.6 

49 

64692 

63898.6 

97879 

101856 

38 

69167.5 

73039.6 

39 

68426.5 

72675.7 

96541.3 

101020 

37 

68997 

73367.5 

41 

65515 

69760.9 

97787.2 

97777.4 

44 

68751.1 

68619.9 

45 

67472.4 

67715 

91193.8 

89928.2 

49 

65357.6 

63967.4 

54 

60610.1 

60492 

91485.6 

90675.7 

48 

65574 

64649.4 

51 

62121.5 

62364.2 

94803.6 

94488.9 

46 

67164.2 

66731.4 

53 

62092 

61095.3 

93295.4 

97095.4 

41 

66518.3 

70203.5 

45 

62646.1 

67590.4 

92308.5 

96275.8 

41 

65908.4 

69760.9 

45 

62229.7 

67174 

96256.1 

100538 

39 

67908.4 

72075.7 

40 

66774 

71324.8 

95098.7 

93843 

46 

67308.4 

65928.1 

49 

63675.6 

63983.8 

84731.4 

87006.9 

49 

62157.6 

64301.9 

52 

59078.9 

62111.7 

97164.3 

95918.4 

45 

68298.6 

66924.8 

50 

63196.9 

63078.9 

99262.6 

101597 

40 

69383.9 

71580.6 

44 

65737.9 

68515 

92452.8 

96924.9 

41 

66019.9 

70377.3 

45 

63351 

67600.2 

97033.1 

95974.1 

45 

68387.1 

67213.4 

47 

65947.8 

66193.7 

92741.3 

95538 

43 

66249.4 

68915 

45 

64259.2 

67039.6 

92154.4 

94669.2 

44 

65859.2 

68242.9 

45 

65046.1 

67819.9 

88334.7 

88292.1 

50 

64026.5 

63865.8 

51 

62885.5 

63128.1 

86242.9 

89006.9 

48 

62856 

65485.5 

49 

61351 

64387.1 

89783.9 

94439.7 

42 

64721.5 

69265.8 

43 

64088.8 

69029.8 

93967.5 

98636.4 

40 

66849.4 

71403.5 

41 

65816.6 

70757.6 

94705.2 

93947.9 

47 

67242.9 

66370.7 

49 

64446.1 

64760.9 

92056.1 

94577.4 

44 

65833 

68223.2 

48 

61846.1 

64892 

95390.5 

95370.8 

45 

67567.5 

67426.5 

48 

65934.7 

65141,2 

98705.3 

101036 

40 

69669.1 

71869.1 

43 

66636.3 

69413.4 

97098.7 

101387 

38 

68419.9 

72593.7 

43 

64865.8 

69115 

96292.1 

100968 

39 

68006.8 

72570.8 

49 

60242.8 

64492 

96629.9 

99443 

40 

68082.2 

70760.9 

43 

66206.8 

68987.1 

90367.5 

92669.2 

45 

64964.2 

67134.7 

48 

61842.8 

64619.9 

87446.2 

86879 

51 

63600.2 

62921.5 

53 

61006.8 

61249.4 

96292.1 

100269 

40 

67915 

71780.6 

44 

63547.8 

68488.8 

93164.3 

92790.5 

47 

66351.1 

65846.1 

51 

62390.4 

62698.6 

93528.2 

93502 

46 

66567.4 

66426.5 

51 

63295.3 

62501.9 

92947.9 

92183.9 

47 

66229.7 

65344.5 

49 

63508.4 

63816.6 

95629.8 

98836.4 

40 

67639.6 

70718.3 

43 

65869.1 

68915 

95764.3 

95492.1 

45 

67688.8 

67298.6 

47 

65351.1 

65593.7 

98177.4 

100505 

41 

68839.6 

71033 

46 

64115 

66892 

96262.6 

100240 

39 

67931.4 

71797 

40 

67301.9 

71551.1 

98420 

97885.6 

44 

69062.5 

68416.6 

49 

64078.9 

64324.8 

98265.9 

101079 

36 

71072.4 

73767.5 

39 

69069.1 

71856 

95600.3 

95328.2 

45 

67518.3 

67131.4 

46 

66219.9 

66213.4 

97469.2 

97524.9 

43 

68537.9 

68469.1 

45 

67475.6 

67429.8 

BASELINE-5 


second  designation  |first  lock 


FINAL  DATA 


second  lock 


launch 


time 

range 

range  to 
airfield 

time 

range 

range  to 
airfield 

time 

range 

range  tc 
airfield 

53.30 

59113 

60752 

56.30 

56818 

58332 

57.30 

56354 

57527 

59.42 

54378 

55871 

4.04 

3879 

3203 

4.04 

2533 

3204 

4.04 

3868 

3225 

3.88 

2308 

3061 

53 

57862.5 

60898.6 

56 

53816.6 

58275.6 

57 

54980.5 

57544.5 

59 

51524.8 

55960.9 

50 

63570.7 

63878.9 

53 

56839.5 

61324.8 

54 

60708.4 

60580.5 

56 

54537.9 

59003.5 

55 

60498.6 

59705.1 

58 

53721.5 

57488.7 

59 

57859.2 

56613.3 

61 

51246.1 

54987.1 

51 

59855.9 

62895.3 

54 

60469.1 

60341.2 

55 

57019.9 

59603.5 

57 

57905.1 

57754.3 

59 

53282.2 

56308.4 

62 

55131.3 

53865.8 

63 

50269 

52777.2 

65 

52665.8 

51377.2 

62 

53462.5 

53829.7 

65 

52623.1 

51337.9 

66 

50764.1 

50610 

68 

50357.6 

49052.6 

52 

57751 

62000.2 

55 

57403.5 

59721.5 

56 

55085.4 

58862.5 

58 

54977.2 

57272.3 

55 

54931.3 

59177.3 

58 

57193.6 

56606.8 

59 

52114.9 

55862.5 

61 

54888.7 

54278.9 

53 

61118.2 

60846.1 

56 

54737.9 

58511.7 

57 

58246.1 

57528.1 

59 

52305.1 

56055.9 

50 

63272.4 

63154.3 

53 

58482.2 

60813.3 

54 

60633 

60078.9 

56 

56174 

58485.5 

58 

54508.4 

57275.6 

61 

54829.7 

54596.9 

62 

51354.3 

53613.3 

64 

52662.5 

52410 

50 

63518.3 

63413.3 

53 

60983.8 

60796.9 

54 

60475.6 

59934.6 

57 

57951 

57741.2 

45 

63803.5 

66849.4 

48 

59937.9 

64446.1 

49 

61259.2 

63878.9 

51 

57862.5 

62354.3 

48 

64534.7 

64836.3 

51 

58639.6 

62442.8 

52 

61846.1 

61724.8 

54 

56282.2 

60069.1 

56 

53718.2 

58662.5 

59 

57511.7 

56269.1 

60 

50983.8 

55419.9 

62 

55121.5 

53855.9 

58 

54321.5 

57347.7 

61 

56016.6 

55059.2 

62 

51633 

54154.3 

64 

53551 

52570.7 

53 

57698.6 

60734.6 

56 

59541.2 

58311.7 

57 

54777.2 

57337.9 

59 

57078.9 

55829.7 

56 

58203.5 

58501.8 

59 

57337.9 

56092 

60 

55410 

55236.3 

62 

54987.1 

53721.5 

48 

62692 

65469.1 

51 

58937.9 

62744.5 

52 

59915 

62259.2 

54 

56819.9 

60610.1 

50 

62872.4 

62751 

53 

56816.6 

60606.8 

54 

60069.1 

59508.4 

56 

54390.4 

58160.9 

49 

60190.4 

64436.3 

52 

62252.7 

62078.9 

53 

57508.4 

61305.1 

57 

57862.5 

57655.9 

61 

51918.2 

54678.9 

64 

52823.1 

52203.5 

65 

49367.4 

51603.5 

67 

50596.9 

49947.7 

59 

52980.5 

55747.7 

62 

53662.5 

53419.9 

63 

50429.7 

52678.9 

65 

51321.5 

51055.9 

60 

55085.4 

55324.8 

63 

54433 

52954.3 

64 

52449.4 

52193.6 

66 

52131.3 

50629.7 

51 

59482.2 

62521.5 

54 

55675.6 

60151 

55 

56836.3 

59416.6 

57 

53492 

57951 

53 

60367.4 

60672.3 

56 

53695.3 

58154.3 

57 

57439.5 

57288.7 

59 

51514.9 

55954.3 

49 

64915 

64121.5 

52 

57554.3 

61652.7 

53 

62105.1 

60892 

55 

55177.2 

59259.2 

57 

54587.1 

57613.3 

60 

55246.1 

55072.3 

61 

51813.3 

54337.9 

63 

52947.7 

52751 

57 

58862.5 

58069.1 

60 

53098.5 

55633 

61 

56137.9 

54882.2 

63 

50918.2 

53429.7 

57 

58295.3 

57501.8 

60 

55452.7 

54855.9 

61 

55246.1 

53983.8 

63 

53213.3 

52593.6 

48 

61167.4 

65416.6 

51 

60485.5 

62829.7 

52 

58036.3 

61836.3 

54 

57908.4 

60229.7 

52 

62160.9 

62046.1 

55 

55855.9 

59639.6 

56 

59410 

58855.9 

58 

53442.8 

57206.8 

52 

57757.6 

62006.8 

55 

59898.6 

59708.4 

56 

55177.2 

58954.3 

58 

57531.4 

57321.5 

49 

64049.4 

63928.1 

52 

59331.4 

61672.3 

53 

61357.6 

60806.8 

55 

56895.3 

59213.3 

56 

58633 

58872.3 

59 

54193.6 

56482.2 

60 

55721.5 

55495.3 

62 

51954.3 

54216.6 

57 

58272.3 

58154.3 

60 

55915 

55688.7 

61 

55554.3 

54964.1 

63 

53596.9 

53351 

57 

53213.3 

58160.9 

60 

53190.4 

55724.8 

61 

50439.5 

54869 

63 

50875.6 

53383.8 

51 

62256 

62560.9 

54 

55787.1 

60265.8 

55 

59524.8 

59393.7 

57 

53367.4 

57823.2 

54 

61128.1 

60334.6 

57 

53393.6 

57852.7 

58 

58252.7 

57013.3 

60 

51098.5 

55531.3 

56 

55970.7 

58993.7 

59 

56587.1 

56433 

60 

53275.6 

55806.8 

62 

54396.9 

54223.1 

55 

59826.4 

59029.7 

58 

54337.9 

56895.3 

59 

57285.4 

56036.3 

61 

51928.1 

54462.5 

56 

58442.8 

58747.8 

59 

57796.9 

56554.3 

60 

55751 

55587.1 

62 

55367.4 

54105.1 

49 

60344.5 

64593.7 

52 

59751 

62088.7 

53 

57685.4 

61482.2 

55 

57531.4 

59849.4 

48 

64918.3 

64803.5 

51 

58708.4 

62515 

52 

62311.7 

61777.3 

54 

56370.7 

60157.6 

53 

61023.2 

61262.5 

56 

55121.5 

58898.6 

57 

58236.3 

58029.7 

59 

52701.8 

56455.9 

50 

63298.6 

63174 

53 

58616.6 

60951 

54 

60433 

59875.6 

56 

56183.8 

58498.6 

54 

59524.8 

59767.4 

57 

55144.5 

57446.1 

58 

56911.7 

56698.6 

60 

52855.9 

55134.6 

60 

55672.3 

55547.7 

63 

53593.6 

53351 

64 

52977.2 

52354.3 

66 

51154.3 

50882.1 

52 

58859.2 

61898.6 

55 

55095.3 

59564.1 

56 

56141.2 

58715 

58 

52806.7 

57255.9 

57 

52924.8 

57872.3 

60 

55541.2 

55374 

61 

50341.2 

54767.4 

63 

53308.4 

53118.2 

55 

54252.7 

59193.7 

58 

57803.5 

56557.6 

59 

51465.8 

55901.8 

61 

55714.9 

54455.9 

56 

55029.7 

58062.5 

59 

55934.6 

55764.1 

60 

52351 

54885.4 

62 

53518.2 

53324.8 

50 

63134.6 

63128.1 

53 

58082.2 

60672.3 

54 

60134.6 

59695.3 

56 

55737.9 

58305.1 

54 

59606.8 

59908.4 

57 

57705.1 

57498.6 

58 

56921.5 

56760.9 

60 

55410 

55180.5 

50 

59354.3 

63603.5 

53 

58931.4 

61262.5 

54 

56646.1 

60433 

56 

56639.5 

58951 

48 

65275.6 

65164.2 

51 

58754.3 

62560.9 

52 

62472.4 

61941.2 

54 

56521.5 

60308.4 

56 

54472.3 

58721.5 

59 

56354.3 

56131.3 

60 

51741.2 

55485.4 

62 

54092 

53852.6 

44 

67888.8 

67770.7 

47 

62996.9 

65367.4 

48 

65170.7 

64649.4 

53 

57878.9 

60206.8 

54 

57062.5 

59833 

57 

57849.4 

57393.6 

58 

54377.2 

56672.3 

60 

55587.1 

55111.7 

50 

63069.1 

63311.7 

53 

61429.7 

60954.3 

54 

60377.3 

60187.1 

57 

67852.7 

57351 
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Appendix  1 .2 

Complete  Set  of  Pixelated  Views  of  KC-1 0  and  B-52 
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Different  levels  of  pixelation  of  a  KC-10  Model  at  an  orientation  of  0  degrees 
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53 


Different  levels  of  pixelation  of  a  KC-1 0  Model  at  an  orientation  of  30  degrees 
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Different  levels  of  pixelation  of  a  B-52  Model  at  an  orientation  of  30  degrees 
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Different  levels  of  pixelation  of  a  KC-10  Model  at  an  orientation  of  60  degrees. 
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Different  levels  of  pixelation  of  a  B-52  Model  at  an  orientation  of  60  degrees. 
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Different  levels  of  pixelation  of  a  KC-10  Model  at  an  orientation  of  90  degrees 
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Different  levels  of  pixelation  of  a  B-52  Model  at  an  orientation  of  90  degrees 
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Different  levels  of  pixelation  of  a  KC-1 0  Model  at  an  orientation  of  -90  degrees. 
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Different  levels  of  pixelation  of  a  B-52  Model  at  an  orientation  of  -90  degrees. 
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Appendix  1 .3 

Selected  Text  Output  from  the  Baseline  Condition 
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Appendix  1 .4 

Selected  Text  Output  from  the  RTIC  Condition 
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Appendix  1 .5 

Snapshot  Of  A  ModSAF  Screen  of  a  Plane  Firing  on  Targets 
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Appendix  1 .6 
ANOVA  Results 
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COCKPIT  BREAKDOWN  1=ENHANCED.  2=BASELINE 


13:28  Tuesday.  October  21.  1997 


CPIT=1 


Variable 

Mean 

Std  Dev  N 

FDT 

13.3250000 

4.8751858  120 

FDR 

91121.83 

3457.02  120 

SDT 

19.4000000 

4.6189234  120 

SDR 

86631.45 

4405.17  120 

SLT 

23.4000000 

4.61S9234  120 

SLR 

83682.29 

4388.57  120 

LT 

54.6083333 

2.6735424  120 

LR 

57901.19  ; 

37.2018016  ,120 

CPIT=2 


Variable 

Mean 

Std  Dev  N 

FDT 

46.5500000 

3.9701618  120 

FDR 

64570.25 

2142.86  120 

SDT 

52.5083533 

3.7S40795  120 

SDR 

59553.24 

3813.64  120 

SLT 

56.5166667 

3.7884906  120 

SLR 

56770.23 

3782.00  120 

LT 

58.6750000 

3.5930898  120 

LR 

55262.55 

2135.15  120 

T.^RGET  COMPLEXITY  BRE.AKDOW^'  3=  3  T.ARGETS.  5-  5  T.ARGETS 


13:28  Tuesday.  October 


TGT=3 


Variable 

Mean 

Std  Dev  N 

FDT 

29.6083333 

17,4497097  120 

FDR 

78322.49 

13682.87  120 

SDT 

34.9833333 

17.2592605  120 

SDR 

73673.15 

14306.77  120 

SLT 

38.9916667 

17.2687583  120 

SLR 

70797.22 

14230.96  120 

LT 

56.3583333 

3.3224367  120 

LR 

57020.20 

1378.07  120 

TGT=5 


\'ariable 

Mean 

Std  Dev  N 

FDT 

30,2666667 

17.0706556  120 

FDR 

77369.59 

13576.42  120 

SDT 

36.9250000 

16.9869760  120 

SDR 

72511.54 

14081.22  120 

SLT 

40.9250000 

16.9869760  120 

SLR 

69655.30 

13984.20  120 

LT 

56.9250000 

4.1468070  120 

LR 

56143.53 

2405.03  120 

1 


INTERACTION  COCKPIT  X  T.^GET 


13:2$  Tuesday.  October  21.  1997  3 


CPIT=1  TGT=3 


Variable 

Mean 

Std  Dev  N 

FDT 

12.7500000 

4.5682211  60 

FDR 

91689.83 

3227.73  60 

SDT 

18.2500000 

4.4707145  60 

SDR 

87352,33 

4285.15  60 

SLT 

22.2500000 

4.4707145  60 

SLR 

84407.62 

4263.20  60 

LT 

54.7833333 

2.7127550  60 

LR 

57893,54 

36.0374301  60 

Variable 

Mean 

Std  Dev  N- 

FDT 

13.9000000 

5.1377631  60 

FDR 

90553.82 

3609.72  60 

SDT 

20.5500000 

4.5114731  60 

SDR 

85910.58 

4440.78  60 

SLT 

24.5500000 

4.5114731  60 

SLR 

82956.96 

4427.25  60 

LT 

54.4333333 

2.6448970  60 

LR 

57908.83 

37.0649392  60 

CPIT=2  TGT=3 


’N'ariable 

Mean 

Std  Dev  N 

FDT 

46.4666667 

3.9033522  60 

FDR 

64955.15 

1941.16  60 

SDT 

51.7166667 

3.3652S47  60 

SDR 

59993.98 

3727.13  60 

SLT 

55.7333333 

3.3792320  60 

SLR 

57186.82 

3678.70  60 

LT 

57.9333333 

3.1400457  60 

LR 

56146.87 

1509.23  60 

INTERACTION  COCKPIT  X  T.^vGET  13:28  Tuesday,  October  21,  1997  4 

- CPIT=2  TGT=5 - 


Variable 

Mean 

std  Dev  N 

FDT 

46.6333333 

4.0670927  60 

FDR 

64185.36 

2278.62  60 

SDT 

53.3000000 

4.0350162  60 

SDR 

59112.51 

3879.14  60 

SLT 

57.3000000 

4.0350162  60 

SLR 

56353.64 

3S6S.30  60 

LT 

59.4166667 

3.SS08162  60 

LR 

54378.22 

2307.97  60 
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ANCVAFOR  FIRST  DESIGNATE  TL\IE 


13:28  Tuesday.  October  21.  1997  5 


Analysis  of  Variance  Procedure 
Class  Level  Information 

Gass  Levels  Values 

CPIT  2  12 

TGT  2  35 


Number  of  observ'ations  in  data  set  =  240 


ANOVA  FOR  FIRST  DESIGNATE  TIME 
Analysis  of  Variance  Procedure 

Dependent  Variable:  FDT 

Source  DF  Sum  of  Squares  Mean  Square 

66274 .54583333  2209 1.51 52':77$ 

4663.51666667  19.'"60663S4 

70938.06250000 

C.V.  Root  MSE 

14.84859  4.44529682 


13:28  Tuesdav.  October  2 


Model  3 

Error  236 

Corrected  Total  239 
R- Square 
0.934259 


F  V alue  Pr  >  F 

1117.95  0.0001 


FDT  Mean 
29.93750000 


Source 

CPIT 

TGT 

cpit-tgt 


DF 

1 

1 


Anova  SS 

66234-03750000 

26.00416667 

14.50416667 


Mean  Square  F  Value 


66234.03750000 

26.0041666*^ 

14.50416667 


3351.81 

1.32 

0.73 


Pr>F 

0.0001 

0.2525 

0.3925 


1.  1997  6 
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ANOVA  FOR  FIRST  DESIGNATE  R.^NGE 


13:28  Tuesday,  October  21. 1997  7 


Analysis  of  Variance  Procedure 
aass  Level  Information 

Gass  Levels  Values 

.G>IT  2  1 2 

TGT  2  35 


Number  of  observ'ations  in  data  set  =  240 


.ANOVA  FOR  FIRST  DESIGNATE  R.ANGE  13:28  Tuesday,  October  2L  1997  8 


Analysis  of  Variance  Procedure 


Dependent  Variable:  FDR 


Source 

DF 

Sum  of  Squares 

Mean  Square  ] 

F  Value  Pr>F 

Model 

3 

42355650185.  n4S0000 

14118550061.72490000 

1742.57  0.0001 

Error 

236 

1912103203,95605000 

8102132.22015278 

Coirected  Total  239 

44267753389.13080000 

R-Square 

C.V. 

Root  MSE 

FDR  Mean 

0.956806 

3.656479 

2846.42446240 

77846.03916667 

Source 

DF 

.•^nova  SS 

Mean  Square  F 

Value  Pr>F 

CPIT  1 

TGT  1 

CP1T"‘TGT  1 

42299157478.20650000 

544S0914.01879SSO 

2011792.94946289 

42299 15747S.20650000 
54480914.01879880 
2011792.94946289 

5220.74  0.0001 

6.72  O.OIOI 

0.25  0.6187 

ANOVA  FOR  SECOND  DESIGNATE  TIME  13:28  Tuesday,  October  21. 1997  9 

Analysis  of  Variance  Procedure 
Class  Level  Information 

Class  Levels  Values 

CPIT  2  1 2 

TGT  235 

Number  of  observations  in  data  set  =  240 

ANOVA  FOR  SECOND  DESIGNATE  TINE  13:28  Tuesday.  October  21.  199"  10 

■Analysis  of  Variance  Procedure 

Dependent  Variable:  SDT 

Source  DF  Sum  of  Squares  Mean  Square  F  Value  Pr>F 

Model  3  66003.61250000  22001.3041666'^  1295.19  0.0001 

Error  256  400S.8S333533  16.9S6793"9 

Corrected  Total  239  "0012.49583333 

R-Square  C.V.  Root  MSE  SDT  Mean 

0,942740  11.46522  4.12150383  35.9541666" 

Source  DF  .Ano\'a  SS  Mean  Square  F  Value  Pr  >  F 

CPIT  1  65T69."04 16667  65769.70416667  58"1.81  0.0001 

TGT  I  226.20416667  226.2041666'^  13.32  0.0005 

CPIT“TGT  1  7.7041666"  7.70416667  0.45  0.5013 
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anovafor  second  designate  range 


13:28  Tuesday,  October  21,  1997  1 1 


Analysis  of  Variance  Procedure 
Class  Level  Information 

Class  Levels  Values 

CPIT  2  12 

TGT  2  35 

Number  of  obser\Titions  in  data  set  =  240 

ANOVA  for  second  designate  range  13:28  Tuesday.  October  21.  1997  12 

.Analysis  of  Variance  Procedure 

Dependent  Variable;  SDR 


Source 

DF 

Sum  of  Squares 

Mean  Square  F  V alue  Pr  >  F 

Model 

3 

44079436580.94600000 

14693145460.31530000 

876.91  0.0001 

Error 

236 

3954306993.26879000 

16755538.10707110 

Corrected  Total  239 

48033743374.21480000 

R-Square 

C.V. 

Root  MSE 

SDR  Mean 

0.9176''6 

5.600248 

4093.35291748 

73092.34666667 

Source 

DF 

.Anova  SS 

Mean  Square  F  Value  Pr  >  F 

CPIT  1 

TGT  1 

CPIT-TGT  1 

43993767408.244 10000 
S0960267. 52465820 
4708705.17724609 

43993767408.244 10000 
80960267.52465820 
4708705.17724609 

2625.63  0.0001 

4.83  0.0289 

0.28  0.5965 

1 


ANOVA  FOR  FIRST  LOCK  TI\IE 


15:10  Tuesday.  October  21.  1997  4 


Analj'sis  of  Variance  Procedure 
Class  Level  Information 

Class  Levels  Values 

CPIT  2  12 

TGT  235 

Number  of  observations  in  data  set  =  240 

FOR  FIRST  LOCK  TIME  15:10  Tuesday.  October  21.  1997  5 

•Analysis  of  Variance  Procedure 

Dependent  Variable:  FLT 

Source  DF  Sum  of  Squares  Mean  Square  F  Value 

Model  3  66003.61250000  22001.20416667  1295.19 

Error  236  4008.8S333333  16.986793"9 

Corrected  Total  239  70012.49583333 

R-Square  C.V.  Root  MSE  FLT  Mean 

0.942740  10.58039  4.12150383  38.95416667 

Source  DF  .AnovaSS  Mean  Square  F  Value  Pr>F 

CPIT  1  65769.70416667  65"69.7W  16667  3871.81  0.0001 

TGT  1  226.20416667  226.20416667  13.32  0.()003 

CPIT“TGT  1  7.70416667  7.70416667  0.45  0.5013 


Pr>F 

0.0001 


78 


1 


ANOV  A  FOR  FIRST  LOCK  R.ANGE 


15:10  Tuesday.  October  21,  1997  6 


Analysis  ofVariance  Procedure 
Clis  Level  Infonnation 

Qass  Levels  Values 

CPIT  2  12 

TGT  235 

Number  of  observations  in  data  set  =  240 

.ANOVA  FOR  FIRST  LOCK  RANGE  15:10  Tuesday,  October  21.  1997 

.Analysis  ofVariance  Procedure 

Dependent  V ariable:  FLR 

Source  DF  Sum  of  Squares  Mean  Square  F  Value  Pr  >  F 

Model  3  4 1792S62254. 16520000  13930954084.72  n0000  1695.58  0.0001 

Error  236  1938987021.95532000  8216046.'"0320052 

Corrected  Total  239  43731849276.12060000 

R-Square  C.V.  Root  MSE  FLR  Mean 

0.955662  4.041129  2866.36471915  70929.79750000 

Source  DF  .Anova  SS  Mean  Square  F  Value  Pr  >  F 

CPIT  1  41556448362.24240000  41556448362.24240000  5057.96  0.0001 

TGT  1  236063475.65600500  236063475.65600500  28.73  0.0001 

CPIT-TGT  1  350416.26684570  350416.26684570  0.04  0.8366 
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1 


ANOVAFOR  SECOND  LOCK  TIME 


13:28  Tuesday.  October  2L  1997  13 


Ajialv*sis  of  Variance  Procedure 
Class  Level  Information 

Gass  Levels  Values 

CPIT  2  1 2 

TGT  235 

Number  of  obsen'ations  in  data  set  =  240 

13:28  Tuesday.  October  21.  1997  14 

F  Value  Pr  >  F 

1294.02  0.0001 


SLT  Mean 
39.95833333 


1  ANOVA  FOR  SECOND  LOCK  TIME 

Analysis  of  Variance  Procedure 

Dependent  Variable:  SLT 

Source  DF  Sum  of  Squares  Mean  Square 

Model  3  66035.15000000  22011."166666- 

Error  236  4014.43333333  17,010310^3 

Corrected  Total  239  70049. 5S333333 

R- Square  C.V.  Root  MSE 

0.942692  10.32164  4.12435580 


Source  DF  Anovz  SS  Mean  Square  F  Value  Pr  >  F 

CPIT  1  65802.8166666-^  65802.81666667  3868.41  0.0001 

TGT  1  224.26666667  224.26666667  13.18  0.0003 

Cpit-'TGT  1  S.06666667  8.06666667  0.47  0.4917 
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1 


ANOVAFOR  SECOND  LOCK  RANGE 


13:28  Tuesday,  October  21. 1997  15 


Anal^'sis  of  Variance  Procedure 
Class  Level  Infonnation 

Gass  Levels  Values 

CPIT  2  1 2 

TGT  2  35 


Number  of  observations  in  data  set  =  240 

1  .^OVA  FOR  SECOND  LOCK  R,^NGE  13:28  Tuesday,  October  2 

.Analysis  of  Variance  Procedure 

Dependent  Variable:  SLR 


Source 

DF 

Sum  of  Squares 

Mean  Square  F 

alue  Pr  >  F 

Model 

3 

43539490896.26140000 

14513163633.08710000 

875.98  0.0001 

Error 

236 

391004S834.SS891000 

■  16568003.53766490 

Corrected  Total  2i 

39  474495390 1.1 5030000 

R-Square 

c.v. 

Root  MSE 

SLR  Mean 

0.917596 

5.^96096 

40'"0.3S125213 

70226.25666667 

Source 

DF 

.Anova  SS 

Mean  Square  F  Value  Pr  >  F 

CPIT  1 

TGT  1 

CPIT*TGT 

43455533024.20500000 

7S23S64S.S0126950 

1  5719223.25512695 

43455533024.20500000 

7S23S64S.80126950 

5719223.25512695 

2622.86  0.0001 
4.72  0.0308 

0.35  0.5574 

1,  1997  16 
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1  ANOVA  FOR  LAUNCH  TIME  13:28  Tuesday,  October  21,  1997  17 

Analysis  of  Variance  Procedure 
Class  Level  Information 

Gass  Levels  Values 

CPIT  2  1 2 

TGT  235 

Number  of  obser\"ations  in  data  set  =  240 

1  ,A\'OVA  FOR  LAL^^H  TIME  13:28  Tuesday.  October  21,  1997  IS 

.dialysis  of  Variance  Procedure 


Dependent  Variable:  LT 

Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Value  Pr  > 

Model 

3 

1061.95000000 

353.98333333 

36.05  0.0001 

Error 

236 

*^317-^333^^^^ 

9.818-8531 

Corrected  Total  239 

3379.18333333 

R- Square 

C.V. 

Root  MSE 

LT  Mean 

0.314262 

5.532136 

3.13349411 

56.64166667 

Source 

DF 

,^iova  SS 

Mean  Square 

F  3*alue  Pr  >  F 

CPIT 

1 

992.26666667 

992.26666667 

101.06  0.0001 

TGT 

1 

19.26666667 

19.26666667 

1.96  0.1626 

CPIT-TGT 

1 

50.41666667 

50.41666667 

5.13  0.0244 
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ANOVAS  FOR  LAUNCH  RANGE 


13:28  Tuesday,  October  21,  1997  19 


Analysis  of  Variance  Procedure 
Class  Level  Information 

Qass  Levels  Values 

.  CPIT  2  1 2 

TGT  2  35 

Number  of  obsen^ons  in  data  set  =  240 


ANOVAS  FOR  L.ALNCH  R.ANGE  13:28  Tuesday.  October  21.  1997  20 


Analysis  of  Variance  Procedure 


Dependent  Variable:  LR 
Source  DF  Sum  of  Sqxiares 

Model  511595693.10827600 

Error  236  44SS22124.327SS000 

Corrected  Total  239  9604^817.43615700 

R-Square  C.V. 

0.532680  2.437271 

Source  DF  .^ova  SS 

CPIT  1  417746054.56872500 

tot  1  461 134 55.67102050 

CPIT’^'TGT  1  47736182.86853020 


Mean  Square  F  Value  Pr  >  F 
170531897.70275800  89.6'^  0.0001 

1901788.66240628 

Root  MSE  LR  Mean 

1379.05353863  5658 1 .86625000 

Mean  Square  F  Value  Pr  >  F 

417746054.56872500  219.66  0.0001 

46113455.67102050  24.25  0.0001 

47736182.86853020  25.10  0.0001 


Appendix  1 .7 
Soar  Source  Code 
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